[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:31 dist-test-slave-w206 krb5kdc[2178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:31 dist-test-slave-w206 krb5kdc[2178](info): set up 2 sockets
Aug 08 10:07:31 dist-test-slave-w206 krb5kdc[2178](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 10:07:32.787386 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:07:32 dist-test-slave-w206 krb5kdc[2178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:07:32.924427 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38537
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:38537
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:33.009289 2192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:33.009531 2192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:33.009555 2192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:33.010735 2192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:33.010788 2192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:33.010802 2192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:33.010815 2192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:33.010828 2192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:33.012408 2192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:38537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:33.012676 2192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:33.012928 2192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:33.018414 2198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:33.018525 2197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:33.018429 2201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:33.018415 2192 server_base.cc:1034] running on GCE node
I20260808 10:07:33.018903 2192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:33.019308 2192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:33.020493 2192 hybrid_clock.cc:648] HybridClock initialized: now 1786183653020434 us; error 72 us; skew 500 ppm
Aug 08 10:07:33 dist-test-slave-w206 krb5kdc[2178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183653, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:33.023317 2192 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:07:33.023821 2192 webserver.cc:460] Webserver started at http://127.2.30.190:40979/ using document root <none> and password file <none>
I20260808 10:07:33.024056 2192 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:33.024127 2192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:33.024289 2192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:33.025475 2192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "36bbeb0d2e8142abb7f02316b39f0fc9"
format_stamp: "Formatted at 2026-08-08 10:07:33 on dist-test-slave-w206"
server_key: "5af9d6f81cbe2b6729608453ec08dde5"
server_key_iv: "dcad9d6528c9c37b06ba8e46a374ae8b"
server_key_version: "encryptionkey@0"
I20260808 10:07:33.025899 2192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "36bbeb0d2e8142abb7f02316b39f0fc9"
format_stamp: "Formatted at 2026-08-08 10:07:33 on dist-test-slave-w206"
server_key: "5af9d6f81cbe2b6729608453ec08dde5"
server_key_iv: "dcad9d6528c9c37b06ba8e46a374ae8b"
server_key_version: "encryptionkey@0"
I20260808 10:07:33.027491 2192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:07:33.028283 2208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:33.028470 2192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:33.028577 2192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "36bbeb0d2e8142abb7f02316b39f0fc9"
format_stamp: "Formatted at 2026-08-08 10:07:33 on dist-test-slave-w206"
server_key: "5af9d6f81cbe2b6729608453ec08dde5"
server_key_iv: "dcad9d6528c9c37b06ba8e46a374ae8b"
server_key_version: "encryptionkey@0"
I20260808 10:07:33.028652 2192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:33.051323 2192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:33.053921 2192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:33.054106 2192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:33.058779 2192 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:38537
I20260808 10:07:33.058913 2261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:38537 every 8 connection(s)
I20260808 10:07:33.059273 2192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:33.060190 2262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:33.060652 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 2192
I20260808 10:07:33.060745 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:07:33.061395 2170 external_mini_cluster.cc:1428] Setting key 70d3fcd23694014d034aae79c622f7cf
I20260808 10:07:33.063086 2262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9: Bootstrap starting.
I20260808 10:07:33.063699 2262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:33.063999 2262 log.cc:824] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:33.064828 2262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9: No bootstrap required, opened a new log
I20260808 10:07:33.066893 2262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bbeb0d2e8142abb7f02316b39f0fc9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38537 } }
I20260808 10:07:33.067122 2262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:33.067152 2262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36bbeb0d2e8142abb7f02316b39f0fc9, State: Initialized, Role: FOLLOWER
I20260808 10:07:33.067343 2262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bbeb0d2e8142abb7f02316b39f0fc9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38537 } }
I20260808 10:07:33.067440 2262 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:33.067485 2262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:33.067534 2262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:33.068219 2262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bbeb0d2e8142abb7f02316b39f0fc9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38537 } }
I20260808 10:07:33.068349 2262 leader_election.cc:302] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36bbeb0d2e8142abb7f02316b39f0fc9; no voters:
I20260808 10:07:33.068559 2262 leader_election.cc:288] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:33.068588 2267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:33.068709 2267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [term 1 LEADER]: Becoming Leader. State: Replica: 36bbeb0d2e8142abb7f02316b39f0fc9, State: Running, Role: LEADER
I20260808 10:07:33.068852 2267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bbeb0d2e8142abb7f02316b39f0fc9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38537 } }
I20260808 10:07:33.068866 2262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:33.069283 2268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36bbeb0d2e8142abb7f02316b39f0fc9. Latest consensus state: current_term: 1 leader_uuid: "36bbeb0d2e8142abb7f02316b39f0fc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bbeb0d2e8142abb7f02316b39f0fc9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38537 } } }
I20260808 10:07:33.069449 2268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:33.069907 2274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:33.070228 2269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36bbeb0d2e8142abb7f02316b39f0fc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36bbeb0d2e8142abb7f02316b39f0fc9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38537 } } }
I20260808 10:07:33.070392 2269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9 [sys.catalog]: This master's current role is: LEADER
Aug 08 10:07:33 dist-test-slave-w206 krb5kdc[2178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:33.070387 2274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:33.076211 2274 catalog_manager.cc:1347] Generated new cluster ID: 6bce8fdb935a4dfd9d5955a4cba3e889
I20260808 10:07:33.076273 2274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:33.079120 2264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:33.062349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37524 (local address 127.2.30.190:38537)
0808 10:07:33.062557 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:33.062568 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:33.062840 (+ 272us) server_negotiation.cc:407] Connection header received
0808 10:07:33.065152 (+ 2312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:33.065161 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:33.065227 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:33.065469 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:33.067252 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.068321 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.070038 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.070667 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.070788 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:33.074921 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:33.074925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:33.074925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:33.074945 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:33.076989 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:33.077456 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:33.077459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:33.077459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:33.077555 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:33.077835 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:33.077836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:33.077836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:33.078089 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:33.078263 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:33.078550 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 10:07:33.078685 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
I20260808 10:07:33.090085 2274 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:33.090706 2274 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:33.101245 2274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36bbeb0d2e8142abb7f02316b39f0fc9: Generated new TSK 0
I20260808 10:07:33.101503 2274 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:07:33.138958 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38537
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:33.230470 2289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:33.230720 2289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:33.230749 2289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:33.232474 2289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:33.232582 2289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:33.232640 2289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:07:33.234390 2289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38537
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:33.234709 2289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:33.235019 2289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:33.240552 2297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:33.240540 2294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:33.240615 2289 server_base.cc:1034] running on GCE node
W20260808 10:07:33.240545 2295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:33.241001 2289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:33.241302 2289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:33.242609 2289 hybrid_clock.cc:648] HybridClock initialized: now 1786183653242565 us; error 56 us; skew 500 ppm
Aug 08 10:07:33 dist-test-slave-w206 krb5kdc[2178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183653, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:33.245612 2289 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:07:33.246062 2289 webserver.cc:460] Webserver started at http://127.2.30.129:36057/ using document root <none> and password file <none>
I20260808 10:07:33.246286 2289 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:33.246330 2289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:33.246425 2289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:33.247617 2289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "9aa13bff3f2d405899486de21de41d17"
format_stamp: "Formatted at 2026-08-08 10:07:33 on dist-test-slave-w206"
server_key: "f92005150228e286bea83ed96665b5ee"
server_key_iv: "bac8cb09d7edfe5affacd811389e1b7d"
server_key_version: "encryptionkey@0"
I20260808 10:07:33.248003 2289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "9aa13bff3f2d405899486de21de41d17"
format_stamp: "Formatted at 2026-08-08 10:07:33 on dist-test-slave-w206"
server_key: "f92005150228e286bea83ed96665b5ee"
server_key_iv: "bac8cb09d7edfe5affacd811389e1b7d"
server_key_version: "encryptionkey@0"
I20260808 10:07:33.249395 2289 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:07:33.250236 2304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:33.250507 2289 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:33.250627 2289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "9aa13bff3f2d405899486de21de41d17"
format_stamp: "Formatted at 2026-08-08 10:07:33 on dist-test-slave-w206"
server_key: "f92005150228e286bea83ed96665b5ee"
server_key_iv: "bac8cb09d7edfe5affacd811389e1b7d"
server_key_version: "encryptionkey@0"
I20260808 10:07:33.250715 2289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:33.266211 2289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:33.268536 2289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:33.268721 2289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:33.269002 2289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:33.269418 2289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:33.269457 2289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:33.269482 2289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:33.269495 2289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:33.275708 2289 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:38987
I20260808 10:07:33.275755 2419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:38987 every 8 connection(s)
I20260808 10:07:33.276180 2289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
Aug 08 10:07:33 dist-test-slave-w206 krb5kdc[2178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183653, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:33.285498 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 2289
I20260808 10:07:33.285609 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:07:33.285825 2170 external_mini_cluster.cc:1428] Setting key d30a2f3f2802c8ac948214f34c4f9fc4
I20260808 10:07:33.286651 2264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:33.277048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:36105 (local address 127.2.30.190:38537)
0808 10:07:33.277102 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:33.277105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:33.277362 (+ 257us) server_negotiation.cc:407] Connection header received
0808 10:07:33.277905 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:33.277906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:33.277919 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:33.277983 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:33.279095 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.279654 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.280821 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.281029 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.281088 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:33.283529 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:33.283530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:33.283530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:33.283538 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:33.284994 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:33.285454 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:33.285456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:33.285457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:33.285527 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:33.285873 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:33.285875 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:33.285875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:33.286031 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:33.286098 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:33.286459 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 10:07:33.286557 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 10:07:33.286882 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:33.277048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38537 (local address 127.2.30.129:36105)
0808 10:07:33.277227 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:33.277271 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:33.277805 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:33.278039 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:33.278043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:33.278730 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:33.278935 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:33.278940 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.279751 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.279752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:33.280739 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:33.280743 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.281109 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.281110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:33.281268 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:33.281927 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:33.281936 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:33.283432 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:33.285089 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:33.285097 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:33.285099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:33.285351 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:33.285653 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:33.285654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:33.285655 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:33.285798 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:33.286111 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:33.286119 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:33.286298 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 10:07:33.286417 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 10:07:33.286604 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
I20260808 10:07:33.287489 2420 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38537
I20260808 10:07:33.287612 2420 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:33.287829 2420 heartbeater.cc:511] Master 127.2.30.190:38537 requested a full tablet report, sending...
I20260808 10:07:33.288453 2225 ts_manager.cc:194] Registered new tserver with Master: 9aa13bff3f2d405899486de21de41d17 (127.2.30.129:38987)
I20260808 10:07:33.289088 2225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:36105
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:07:33.317884 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38537
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:33.408564 2426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:33.408856 2426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:33.408901 2426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:33.410106 2426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:33.410259 2426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:33.410316 2426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:07:33.411909 2426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38537
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:33.412240 2426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:33.412510 2426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:33.418144 2432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:33.418154 2434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:33.418154 2431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:33.418604 2426 server_base.cc:1034] running on GCE node
I20260808 10:07:33.418890 2426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:33.419212 2426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:33.420383 2426 hybrid_clock.cc:648] HybridClock initialized: now 1786183653420361 us; error 41 us; skew 500 ppm
Aug 08 10:07:33 dist-test-slave-w206 krb5kdc[2178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183653, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:33.423254 2426 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:07:33.423699 2426 webserver.cc:460] Webserver started at http://127.2.30.130:41443/ using document root <none> and password file <none>
I20260808 10:07:33.423985 2426 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:33.424036 2426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:33.424137 2426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:33.425302 2426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "5d59809ff8f14abf9e5470c5f65feb02"
format_stamp: "Formatted at 2026-08-08 10:07:33 on dist-test-slave-w206"
server_key: "50e26a056e5a53f92f8997be991ca5c0"
server_key_iv: "49fec0c2f29fa53009207a6bd34dc80f"
server_key_version: "encryptionkey@0"
I20260808 10:07:33.425697 2426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "5d59809ff8f14abf9e5470c5f65feb02"
format_stamp: "Formatted at 2026-08-08 10:07:33 on dist-test-slave-w206"
server_key: "50e26a056e5a53f92f8997be991ca5c0"
server_key_iv: "49fec0c2f29fa53009207a6bd34dc80f"
server_key_version: "encryptionkey@0"
I20260808 10:07:33.427335 2426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:07:33.428149 2441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:33.428356 2426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:07:33.428478 2426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "5d59809ff8f14abf9e5470c5f65feb02"
format_stamp: "Formatted at 2026-08-08 10:07:33 on dist-test-slave-w206"
server_key: "50e26a056e5a53f92f8997be991ca5c0"
server_key_iv: "49fec0c2f29fa53009207a6bd34dc80f"
server_key_version: "encryptionkey@0"
I20260808 10:07:33.428558 2426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:33.451671 2426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:33.453950 2426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:33.454126 2426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:33.454401 2426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:33.454741 2426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:33.454798 2426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:33.454881 2426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:33.454912 2426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:33.461532 2426 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:34743
I20260808 10:07:33.461616 2556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:34743 every 8 connection(s)
I20260808 10:07:33.461980 2426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:07:33.464550 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 2426
I20260808 10:07:33.464643 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:07:33.464831 2170 external_mini_cluster.cc:1428] Setting key 7ac8402f447079d305a3bd94b3368fea
Aug 08 10:07:33 dist-test-slave-w206 krb5kdc[2178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183653, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:33.474117 2264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:33.462796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:39869 (local address 127.2.30.190:38537)
0808 10:07:33.462888 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:33.462892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:33.463213 (+ 321us) server_negotiation.cc:407] Connection header received
0808 10:07:33.463698 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:33.463700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:33.463713 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:33.463784 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:33.464819 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.465458 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.467197 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.467403 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.467465 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:33.470475 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:33.470477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:33.470478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:33.470488 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:33.472546 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:33.473006 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:33.473008 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:33.473008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:33.473080 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:33.473344 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:33.473345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:33.473346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:33.473519 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:33.473597 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:33.473893 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 10:07:33.473991 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:07:33.474159 2559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:33.462776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38537 (local address 127.2.30.130:39869)
0808 10:07:33.463036 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:33.463089 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:33.463584 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:33.463858 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:33.463861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:33.464458 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:33.464710 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:33.464716 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.465582 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.465582 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:33.467104 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:33.467107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.467496 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.467497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:33.467665 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:33.468348 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:33.468357 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:33.470328 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:33.472647 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:33.472654 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:33.472657 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:33.472921 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:33.473150 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:33.473150 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:33.473151 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:33.473261 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:33.473590 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:33.473596 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:33.473726 (+ 130us) client_negotiation.cc:769] Sending connection context
0808 10:07:33.473838 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 10:07:33.473975 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":133,"threads_started":1}
I20260808 10:07:33.475090 2557 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38537
I20260808 10:07:33.475215 2557 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:33.475435 2557 heartbeater.cc:511] Master 127.2.30.190:38537 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
I20260808 10:07:33.476075 2225 ts_manager.cc:194] Registered new tserver with Master: 5d59809ff8f14abf9e5470c5f65feb02 (127.2.30.130:34743)
I20260808 10:07:33.476488 2225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:39869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:07:33.498080 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38537
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:32819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:33.585721 2562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:33.585990 2562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:33.586036 2562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:33.587356 2562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:33.587421 2562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:33.587468 2562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:07:33.589016 2562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:32819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38537
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:33.589295 2562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:33.589550 2562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:33.594949 2571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:33.594959 2569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:33.594954 2568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:33.595599 2562 server_base.cc:1034] running on GCE node
I20260808 10:07:33.595889 2562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:33.596151 2562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:33.597332 2562 hybrid_clock.cc:648] HybridClock initialized: now 1786183653597306 us; error 43 us; skew 500 ppm
Aug 08 10:07:33 dist-test-slave-w206 krb5kdc[2178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183653, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:33.600127 2562 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:07:33.600839 2562 webserver.cc:460] Webserver started at http://127.2.30.131:43843/ using document root <none> and password file <none>
I20260808 10:07:33.601128 2562 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:33.601223 2562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:33.601377 2562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:33.602545 2562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "b2777246b20e43a6ad053dc6b2eacb39"
format_stamp: "Formatted at 2026-08-08 10:07:33 on dist-test-slave-w206"
server_key: "61ba3017a749cdd0ae4e60d564aeef1b"
server_key_iv: "1f501fc74ca78dd58c938420b85beb52"
server_key_version: "encryptionkey@0"
I20260808 10:07:33.602936 2562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "b2777246b20e43a6ad053dc6b2eacb39"
format_stamp: "Formatted at 2026-08-08 10:07:33 on dist-test-slave-w206"
server_key: "61ba3017a749cdd0ae4e60d564aeef1b"
server_key_iv: "1f501fc74ca78dd58c938420b85beb52"
server_key_version: "encryptionkey@0"
I20260808 10:07:33.604396 2562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:07:33.605407 2579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:33.605620 2562 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:07:33.605751 2562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "b2777246b20e43a6ad053dc6b2eacb39"
format_stamp: "Formatted at 2026-08-08 10:07:33 on dist-test-slave-w206"
server_key: "61ba3017a749cdd0ae4e60d564aeef1b"
server_key_iv: "1f501fc74ca78dd58c938420b85beb52"
server_key_version: "encryptionkey@0"
I20260808 10:07:33.605834 2562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:33.629597 2562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:33.632519 2562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:33.632716 2562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:33.633087 2562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:33.633453 2562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:33.633512 2562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:33.633558 2562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:33.633584 2562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:33.641170 2562 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:40683
I20260808 10:07:33.641211 2694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:40683 every 8 connection(s)
I20260808 10:07:33.641571 2562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:07:33.644716 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 2562
I20260808 10:07:33.644811 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:07:33.644987 2170 external_mini_cluster.cc:1428] Setting key 4b901a3d8d63e7fa84644aff4e84c531
Aug 08 10:07:33 dist-test-slave-w206 krb5kdc[2178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183653, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:33.652325 2264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:33.642616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:57367 (local address 127.2.30.190:38537)
0808 10:07:33.642694 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:33.642696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:33.643040 (+ 344us) server_negotiation.cc:407] Connection header received
0808 10:07:33.643527 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:33.643528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:33.643540 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:33.643607 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:33.644601 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.645374 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.646606 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.646828 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.646892 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:33.649245 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:33.649246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:33.649246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:33.649255 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:33.650758 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:33.651285 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:33.651287 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:33.651287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:33.651351 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:33.651621 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:33.651622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:33.651622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:33.651761 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:33.651841 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:33.652136 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 10:07:33.652224 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:07:33.652544 2698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:33.642599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38537 (local address 127.2.30.131:57367)
0808 10:07:33.642783 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:33.642853 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:33.643425 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:33.643664 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:33.643667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:33.644282 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:33.644487 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:33.644492 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.645464 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.645465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:33.646521 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:33.646524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.646911 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.646912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:33.647087 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:33.647614 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:33.647622 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:33.649141 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:33.650868 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:33.650876 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:33.650878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:33.651185 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:33.651421 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:33.651422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:33.651422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:33.651545 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:33.651829 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:33.651837 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:33.651993 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 10:07:33.652101 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 10:07:33.652286 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
I20260808 10:07:33.653087 2695 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38537
I20260808 10:07:33.653208 2695 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:33.653425 2695 heartbeater.cc:511] Master 127.2.30.190:38537 requested a full tablet report, sending...
I20260808 10:07:33.653883 2225 ts_manager.cc:194] Registered new tserver with Master: b2777246b20e43a6ad053dc6b2eacb39 (127.2.30.131:40683)
I20260808 10:07:33.654285 2225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:57367
I20260808 10:07:33.657544 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:07:33 dist-test-slave-w206 krb5kdc[2178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183653, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:07:33 dist-test-slave-w206 krb5kdc[2178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183653, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.30.129@KRBTEST.COM
I20260808 10:07:33.683591 2703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:33.674578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41462 (local address 127.2.30.129:38987)
0808 10:07:33.674771 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:33.674776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:33.674788 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:07:33.674815 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:33.674817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:33.674831 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:33.674899 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:33.675606 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.676338 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.677188 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.677545 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.677605 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:33.680322 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:33.680331 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:33.680331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:33.680338 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:33.682114 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:33.682527 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:33.682530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:33.682530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:33.682590 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:33.682870 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:33.682872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:33.682872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:33.683103 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:33.683229 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:33.683432 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 10:07:33.683507 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
W20260808 10:07:33.684062 2333 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41462
Aug 08 10:07:33 dist-test-slave-w206 krb5kdc[2178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:07:33 dist-test-slave-w206 krb5kdc[2178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.129@KRBTEST.COM
I20260808 10:07:33.710219 2703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:33.701892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41470 (local address 127.2.30.129:38987)
0808 10:07:33.701964 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:33.701966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:33.702014 (+ 48us) server_negotiation.cc:407] Connection header received
0808 10:07:33.702067 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:33.702068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:33.702078 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:33.702139 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:33.703080 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.703648 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.704588 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:33.704837 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:33.704907 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:33.707488 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:33.707489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:33.707489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:33.707496 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:33.709033 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:33.709435 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:33.709436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:33.709437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:33.709493 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:33.709784 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:33.709785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:33.709785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:33.709893 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:33.709994 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:33.710072 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 10:07:33.710142 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:07:33.710882 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 2289
I20260808 10:07:33.716609 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 2426
I20260808 10:07:33.723982 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 2562
I20260808 10:07:33.729966 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 2192
2026-08-08T10:07:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2058 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:33 dist-test-slave-w206 krb5kdc[2710](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:33 dist-test-slave-w206 krb5kdc[2710](info): set up 2 sockets
Aug 08 10:07:33 dist-test-slave-w206 krb5kdc[2710](info): commencing operation
krb5kdc: starting...
W20260808 10:07:34.818128 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:07:34 dist-test-slave-w206 krb5kdc[2710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:07:34.954401 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:42165
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:42165
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:35.041326 2723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:35.041599 2723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:35.041639 2723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:35.042822 2723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:35.042897 2723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:35.042922 2723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:35.042946 2723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:35.042999 2723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:35.044584 2723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:42165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:42165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:35.044891 2723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:35.045176 2723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:35.050335 2730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:35.050323 2733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:35.050343 2731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:35.050550 2723 server_base.cc:1034] running on GCE node
I20260808 10:07:35.050872 2723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:35.051208 2723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:35.052392 2723 hybrid_clock.cc:648] HybridClock initialized: now 1786183655052388 us; error 62 us; skew 500 ppm
Aug 08 10:07:35 dist-test-slave-w206 krb5kdc[2710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183655, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:35.055059 2723 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:07:35.055574 2723 webserver.cc:460] Webserver started at http://127.2.30.190:33535/ using document root <none> and password file <none>
I20260808 10:07:35.055825 2723 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:35.055904 2723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:35.056066 2723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:35.057210 2723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "6b4ee16bc52e45398450fe3b165eb640"
format_stamp: "Formatted at 2026-08-08 10:07:35 on dist-test-slave-w206"
server_key: "983803533a30783f3a8799cf55ec7b7f"
server_key_iv: "69e0f62ecf87fde1c2d89ad6a4b11c3e"
server_key_version: "encryptionkey@0"
I20260808 10:07:35.057586 2723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "6b4ee16bc52e45398450fe3b165eb640"
format_stamp: "Formatted at 2026-08-08 10:07:35 on dist-test-slave-w206"
server_key: "983803533a30783f3a8799cf55ec7b7f"
server_key_iv: "69e0f62ecf87fde1c2d89ad6a4b11c3e"
server_key_version: "encryptionkey@0"
I20260808 10:07:35.059082 2723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:07:35.059965 2740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:35.060146 2723 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:35.060247 2723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "6b4ee16bc52e45398450fe3b165eb640"
format_stamp: "Formatted at 2026-08-08 10:07:35 on dist-test-slave-w206"
server_key: "983803533a30783f3a8799cf55ec7b7f"
server_key_iv: "69e0f62ecf87fde1c2d89ad6a4b11c3e"
server_key_version: "encryptionkey@0"
I20260808 10:07:35.060339 2723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:35.082412 2723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:35.084942 2723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:35.085129 2723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:35.089542 2723 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:42165
I20260808 10:07:35.089581 2796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:42165 every 8 connection(s)
I20260808 10:07:35.089998 2723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:35.090580 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 2723
I20260808 10:07:35.090706 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:07:35.090898 2170 external_mini_cluster.cc:1428] Setting key b2122979101a521510adb3e57fc65155
I20260808 10:07:35.091359 2797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:35.095350 2797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640: Bootstrap starting.
I20260808 10:07:35.096191 2797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:35.096580 2797 log.cc:824] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:35.097515 2797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640: No bootstrap required, opened a new log
Aug 08 10:07:35 dist-test-slave-w206 krb5kdc[2710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:35.100013 2797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b4ee16bc52e45398450fe3b165eb640" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 42165 } }
I20260808 10:07:35.100253 2797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:35.100309 2797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b4ee16bc52e45398450fe3b165eb640, State: Initialized, Role: FOLLOWER
I20260808 10:07:35.100579 2797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b4ee16bc52e45398450fe3b165eb640" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 42165 } }
I20260808 10:07:35.100711 2797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:35.100762 2797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:35.100814 2797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:35.101471 2797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b4ee16bc52e45398450fe3b165eb640" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 42165 } }
I20260808 10:07:35.101608 2797 leader_election.cc:302] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b4ee16bc52e45398450fe3b165eb640; no voters:
I20260808 10:07:35.101819 2797 leader_election.cc:288] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:35.101890 2802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:35.102058 2802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [term 1 LEADER]: Becoming Leader. State: Replica: 6b4ee16bc52e45398450fe3b165eb640, State: Running, Role: LEADER
I20260808 10:07:35.102156 2797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:35.102159 2802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b4ee16bc52e45398450fe3b165eb640" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 42165 } }
I20260808 10:07:35.102586 2802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b4ee16bc52e45398450fe3b165eb640. Latest consensus state: current_term: 1 leader_uuid: "6b4ee16bc52e45398450fe3b165eb640" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b4ee16bc52e45398450fe3b165eb640" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 42165 } } }
I20260808 10:07:35.102682 2802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:35.102633 2804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b4ee16bc52e45398450fe3b165eb640" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b4ee16bc52e45398450fe3b165eb640" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 42165 } } }
I20260808 10:07:35.102895 2804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:35.102926 2808 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:35.104225 2800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.091543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55618 (local address 127.2.30.190:42165)
0808 10:07:35.091765 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:35.091774 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:35.091830 (+ 56us) server_negotiation.cc:407] Connection header received
0808 10:07:35.092606 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:35.092616 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:35.092676 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:35.092834 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:35.093739 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.094720 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.095968 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.096533 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.096691 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.099517 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:35.099525 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:35.099525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:35.099546 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:35.102118 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:35.102578 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:35.102582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:35.102582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:35.102728 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:35.103006 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:35.103007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:35.103008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:35.103361 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:35.103524 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:35.103818 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 10:07:35.103977 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":91,"threads_started":1}
I20260808 10:07:35.104050 2808 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:35.106153 2808 catalog_manager.cc:1347] Generated new cluster ID: 99ccc826384c445d9c96d20aee32de61
I20260808 10:07:35.106209 2808 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:35.129380 2808 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:35.129990 2808 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:35.143126 2808 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b4ee16bc52e45398450fe3b165eb640: Generated new TSK 0
I20260808 10:07:35.143405 2808 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:07:35.182898 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:42165
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:35.271067 2823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:35.271296 2823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:35.271324 2823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:35.272555 2823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:35.272599 2823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:35.272626 2823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:07:35.274211 2823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:42165
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:35.274504 2823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:35.274763 2823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:35.280316 2828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:35.280316 2831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:35.280382 2829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:35.280560 2823 server_base.cc:1034] running on GCE node
I20260808 10:07:35.280828 2823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:35.281095 2823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:35.282255 2823 hybrid_clock.cc:648] HybridClock initialized: now 1786183655282224 us; error 54 us; skew 500 ppm
Aug 08 10:07:35 dist-test-slave-w206 krb5kdc[2710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183655, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:35.285146 2823 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:07:35.285734 2823 webserver.cc:460] Webserver started at http://127.2.30.129:44335/ using document root <none> and password file <none>
I20260808 10:07:35.285995 2823 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:35.286067 2823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:35.286226 2823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:35.287469 2823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "a2fba5b178c34ecaab25ac7b2d3727c1"
format_stamp: "Formatted at 2026-08-08 10:07:35 on dist-test-slave-w206"
server_key: "7b4b453bfeedff260f0c819ba7819a8f"
server_key_iv: "629115684a7e2ad20e6bdcb81163db48"
server_key_version: "encryptionkey@0"
I20260808 10:07:35.287848 2823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "a2fba5b178c34ecaab25ac7b2d3727c1"
format_stamp: "Formatted at 2026-08-08 10:07:35 on dist-test-slave-w206"
server_key: "7b4b453bfeedff260f0c819ba7819a8f"
server_key_iv: "629115684a7e2ad20e6bdcb81163db48"
server_key_version: "encryptionkey@0"
I20260808 10:07:35.289426 2823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:07:35.290266 2839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:35.290489 2823 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:35.290601 2823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "a2fba5b178c34ecaab25ac7b2d3727c1"
format_stamp: "Formatted at 2026-08-08 10:07:35 on dist-test-slave-w206"
server_key: "7b4b453bfeedff260f0c819ba7819a8f"
server_key_iv: "629115684a7e2ad20e6bdcb81163db48"
server_key_version: "encryptionkey@0"
I20260808 10:07:35.290679 2823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:35.309962 2823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:35.312338 2823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:35.312523 2823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:35.312798 2823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:35.313144 2823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:35.313200 2823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:35.313247 2823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:35.313287 2823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:35.319473 2823 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:41095
I20260808 10:07:35.319516 2956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:41095 every 8 connection(s)
I20260808 10:07:35.319865 2823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
Aug 08 10:07:35 dist-test-slave-w206 krb5kdc[2710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183655, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:35.329273 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 2823
I20260808 10:07:35.329381 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:07:35.329596 2170 external_mini_cluster.cc:1428] Setting key 51616f11d4c7d50c2526abb18dabb0a5
I20260808 10:07:35.330601 2800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.320823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:36639 (local address 127.2.30.190:42165)
0808 10:07:35.320908 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:35.320911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:35.321062 (+ 151us) server_negotiation.cc:407] Connection header received
0808 10:07:35.322184 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:35.322185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:35.322231 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:35.322296 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:35.322953 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.323554 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.324629 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.324854 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.324925 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.327465 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:35.327466 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:35.327467 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:35.327478 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:35.328881 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:35.329415 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:35.329417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:35.329417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:35.329492 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:35.329835 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:35.329837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:35.329837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:35.329990 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:35.330079 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:35.330413 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 10:07:35.330507 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 10:07:35.330819 2960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.320746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:42165 (local address 127.2.30.129:36639)
0808 10:07:35.320934 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:35.320974 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:35.322045 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:35.322377 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:35.322381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:35.322660 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:35.322862 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:35.322866 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.323649 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.323650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:35.324528 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:35.324530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.324961 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.324962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:35.325092 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.325742 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:35.325751 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:35.327349 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:35.328999 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:35.329008 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:35.329011 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:35.329301 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:35.329578 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:35.329579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:35.329579 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:35.329760 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:35.330070 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:35.330077 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:35.330266 (+ 189us) client_negotiation.cc:769] Sending connection context
0808 10:07:35.330391 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 10:07:35.330561 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":100,"threads_started":1}
I20260808 10:07:35.331352 2957 heartbeater.cc:347] Connected to a master server at 127.2.30.190:42165
I20260808 10:07:35.331470 2957 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:35.331681 2957 heartbeater.cc:511] Master 127.2.30.190:42165 requested a full tablet report, sending...
I20260808 10:07:35.332309 2758 ts_manager.cc:194] Registered new tserver with Master: a2fba5b178c34ecaab25ac7b2d3727c1 (127.2.30.129:41095)
I20260808 10:07:35.333262 2758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:36639
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:07:35.359166 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:42165
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:35.445580 2964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:35.445819 2964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:35.445847 2964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:35.447175 2964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:35.447221 2964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:35.447252 2964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:07:35.448912 2964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:42165
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:35.449214 2964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:35.449474 2964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:35.455050 2972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:35.455041 2970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:35.455189 2964 server_base.cc:1034] running on GCE node
W20260808 10:07:35.455036 2969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:35.455644 2964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:35.455891 2964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:35.457036 2964 hybrid_clock.cc:648] HybridClock initialized: now 1786183655457026 us; error 33 us; skew 500 ppm
Aug 08 10:07:35 dist-test-slave-w206 krb5kdc[2710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183655, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:35.459849 2964 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:07:35.460361 2964 webserver.cc:460] Webserver started at http://127.2.30.130:35667/ using document root <none> and password file <none>
I20260808 10:07:35.460582 2964 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:35.460625 2964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:35.460721 2964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:35.461764 2964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "8c3cb09400a54fb689f399164117fbf3"
format_stamp: "Formatted at 2026-08-08 10:07:35 on dist-test-slave-w206"
server_key: "9e58f07943163b6649083d6a4f174800"
server_key_iv: "a0b7063e5d786957475bd18bc01b6cc6"
server_key_version: "encryptionkey@0"
I20260808 10:07:35.462074 2964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "8c3cb09400a54fb689f399164117fbf3"
format_stamp: "Formatted at 2026-08-08 10:07:35 on dist-test-slave-w206"
server_key: "9e58f07943163b6649083d6a4f174800"
server_key_iv: "a0b7063e5d786957475bd18bc01b6cc6"
server_key_version: "encryptionkey@0"
I20260808 10:07:35.463567 2964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:07:35.464432 2979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:35.464615 2964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:35.464725 2964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "8c3cb09400a54fb689f399164117fbf3"
format_stamp: "Formatted at 2026-08-08 10:07:35 on dist-test-slave-w206"
server_key: "9e58f07943163b6649083d6a4f174800"
server_key_iv: "a0b7063e5d786957475bd18bc01b6cc6"
server_key_version: "encryptionkey@0"
I20260808 10:07:35.464807 2964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:35.483877 2964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:35.486538 2964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:35.486726 2964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:35.487038 2964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:35.487404 2964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:35.487459 2964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:35.487504 2964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:35.487531 2964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:35.493852 2964 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:38743
I20260808 10:07:35.493920 3096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:38743 every 8 connection(s)
I20260808 10:07:35.494294 2964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:07:35.495548 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 2964
I20260808 10:07:35.495653 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:07:35.495814 2170 external_mini_cluster.cc:1428] Setting key b472da53693c114c63221740653d622a
Aug 08 10:07:35 dist-test-slave-w206 krb5kdc[2710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183655, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
I20260808 10:07:35.505086 2800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.495235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:44891 (local address 127.2.30.190:42165)
0808 10:07:35.495304 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:35.495306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:35.495558 (+ 252us) server_negotiation.cc:407] Connection header received
0808 10:07:35.496824 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:35.496825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:35.496837 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:35.496896 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:35.497577 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.498088 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.499176 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.499381 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.499469 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.501826 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:35.501827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:35.501827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:35.501834 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:35.503545 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:35.504013 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:35.504014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:35.504015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:35.504086 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:35.504363 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:35.504364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:35.504365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:35.504503 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:35.504579 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:35.504888 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 10:07:35.504985 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:07:35.505304 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.495225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:42165 (local address 127.2.30.130:44891)
0808 10:07:35.495423 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:35.495461 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:35.496712 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:35.496969 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:35.496973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:35.497265 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:35.497476 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:35.497480 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.498187 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.498189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:35.499090 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:35.499092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.499481 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.499482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:35.499612 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.500203 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:35.500212 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:35.501671 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:35.503651 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:35.503660 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:35.503663 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:35.503928 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:35.504156 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:35.504157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:35.504157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:35.504284 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:35.504579 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:35.504588 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:35.504747 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 10:07:35.504862 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 10:07:35.505041 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":71,"threads_started":1}
I20260808 10:07:35.505810 3097 heartbeater.cc:347] Connected to a master server at 127.2.30.190:42165
I20260808 10:07:35.505955 3097 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:35.506189 3097 heartbeater.cc:511] Master 127.2.30.190:42165 requested a full tablet report, sending...
I20260808 10:07:35.506654 2758 ts_manager.cc:194] Registered new tserver with Master: 8c3cb09400a54fb689f399164117fbf3 (127.2.30.130:38743)
I20260808 10:07:35.507107 2758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:44891
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:07:35.526625 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:42165
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:33053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:35.616122 3102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:35.616384 3102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:35.616415 3102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:35.617712 3102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:35.617754 3102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:35.617785 3102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:07:35.619472 3102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:33053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:42165
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:35.619699 3102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:35.619930 3102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:35.625197 3107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:35.625330 3110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:35.625356 3108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:35.625476 3102 server_base.cc:1034] running on GCE node
I20260808 10:07:35.625734 3102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:35.626025 3102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:35.627175 3102 hybrid_clock.cc:648] HybridClock initialized: now 1786183655627151 us; error 39 us; skew 500 ppm
Aug 08 10:07:35 dist-test-slave-w206 krb5kdc[2710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183655, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:35.629858 3102 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:07:35.630342 3102 webserver.cc:460] Webserver started at http://127.2.30.131:37381/ using document root <none> and password file <none>
I20260808 10:07:35.630599 3102 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:35.630673 3102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:35.630842 3102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:35.632054 3102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "46838f9dc44745f8988268d65c2e5a2a"
format_stamp: "Formatted at 2026-08-08 10:07:35 on dist-test-slave-w206"
server_key: "ee85e962f9488343b7532d1d358704e5"
server_key_iv: "4687c91782e6399651789cbc69aa7d70"
server_key_version: "encryptionkey@0"
I20260808 10:07:35.632436 3102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "46838f9dc44745f8988268d65c2e5a2a"
format_stamp: "Formatted at 2026-08-08 10:07:35 on dist-test-slave-w206"
server_key: "ee85e962f9488343b7532d1d358704e5"
server_key_iv: "4687c91782e6399651789cbc69aa7d70"
server_key_version: "encryptionkey@0"
I20260808 10:07:35.633962 3102 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:07:35.634719 3117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:35.634897 3102 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:07:35.635021 3102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "46838f9dc44745f8988268d65c2e5a2a"
format_stamp: "Formatted at 2026-08-08 10:07:35 on dist-test-slave-w206"
server_key: "ee85e962f9488343b7532d1d358704e5"
server_key_iv: "4687c91782e6399651789cbc69aa7d70"
server_key_version: "encryptionkey@0"
I20260808 10:07:35.635119 3102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:35.652344 3102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:35.654388 3102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:35.654573 3102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:35.654841 3102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:35.655237 3102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:35.655288 3102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:35.655339 3102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:35.655360 3102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:35.662212 3102 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:39733
I20260808 10:07:35.662278 3236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:39733 every 8 connection(s)
I20260808 10:07:35.662601 3102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:07:35.663287 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 3102
I20260808 10:07:35.663434 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:07:35.663659 2170 external_mini_cluster.cc:1428] Setting key c4afc348d362a9699d7907371fad2ecf
Aug 08 10:07:35 dist-test-slave-w206 krb5kdc[2710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183655, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:35.672760 2800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.663841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:53893 (local address 127.2.30.190:42165)
0808 10:07:35.663895 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:35.663897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:35.663951 (+ 54us) server_negotiation.cc:407] Connection header received
0808 10:07:35.665139 (+ 1188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:35.665140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:35.665150 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:35.665196 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:35.665856 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.666342 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.667435 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.667622 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.667718 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.669991 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:35.669992 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:35.669993 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:35.670001 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:35.671310 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:35.671731 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:35.671732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:35.671733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:35.671799 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:35.672061 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:35.672062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:35.672063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:35.672192 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:35.672271 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:35.672571 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 10:07:35.672659 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:07:35.672971 3239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.663516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:42165 (local address 127.2.30.131:53893)
0808 10:07:35.663725 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:35.663781 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:35.665048 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:35.665300 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:35.665304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:35.665580 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:35.665780 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:35.665784 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.666423 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.666424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:35.667358 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:35.667361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.667732 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.667734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:35.667925 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.668436 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:35.668445 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:35.669887 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:35.671409 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:35.671417 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:35.671419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:35.671650 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:35.671876 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:35.671876 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:35.671877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:35.671991 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:35.672261 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:35.672269 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:35.672429 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 10:07:35.672548 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 10:07:35.672717 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":98,"threads_started":1}
I20260808 10:07:35.673559 3237 heartbeater.cc:347] Connected to a master server at 127.2.30.190:42165
I20260808 10:07:35.673650 3237 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:35.673849 3237 heartbeater.cc:511] Master 127.2.30.190:42165 requested a full tablet report, sending...
I20260808 10:07:35.674248 2758 ts_manager.cc:194] Registered new tserver with Master: 46838f9dc44745f8988268d65c2e5a2a (127.2.30.131:39733)
I20260808 10:07:35.674652 2758 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:53893
I20260808 10:07:35.675511 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:07:35.683264 2800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.677265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50426 (local address 127.2.30.190:42165)
0808 10:07:35.677303 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:35.677305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:35.677438 (+ 133us) server_negotiation.cc:407] Connection header received
0808 10:07:35.677516 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:35.677517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:35.677526 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:35.677599 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:35.678167 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.678649 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.679447 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.679644 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.679715 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.680924 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:35.680926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:35.680926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:35.680933 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:35.682051 (+ 1118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:35.682396 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:35.682397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:35.682397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:35.682459 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:35.682663 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:35.682664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:35.682665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:35.682830 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:35.682941 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:35.683072 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 10:07:35.683156 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 10:07:35.685443 2758 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:07:35.687254 2758 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:07:35.694267 3249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.690017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:38743 (local address 127.0.0.1:42978)
0808 10:07:35.690410 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:35.690418 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:35.690548 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:35.690843 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:35.690844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:35.690846 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:35.690894 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:35.690895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.691806 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.691807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:35.693096 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:35.693098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.693943 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.693944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:35.694030 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.694047 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 10:07:35.694119 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 10:07:35.694170 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":79,"threads_started":1}
I20260808 10:07:35.694289 3253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.690153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42978 (local address 127.2.30.130:38743)
0808 10:07:35.690354 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:35.690359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:35.690481 (+ 122us) server_negotiation.cc:407] Connection header received
0808 10:07:35.690643 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:35.690646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:35.690662 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:35.690733 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:35.690993 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.691695 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.693191 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.693781 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.693934 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.694010 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:35.694162 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 10:07:35.694206 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":77,"threads_started":1}
I20260808 10:07:35.695143 3028 tablet_service.cc:1467] Processing CreateTablet for tablet 45db96349afb4cb9b36c6e2cef5b9d14 (DEFAULT_TABLE table=test-table [id=a94a2dcea18a4550acdfbe0be5c37b4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:35.695434 3252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.689782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:41095 (local address 127.0.0.1:35574)
0808 10:07:35.690163 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:35.690209 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:35.690329 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:35.690609 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:35.690612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:35.690615 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:35.690709 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:35.690711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.692068 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.692069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:35.694161 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:35.694163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.695181 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.695182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:35.695246 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.695250 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:07:35.695304 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 10:07:35.695355 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":9,"spinlock_wait_cycles":384,"thread_start_us":96,"threads_started":1}
I20260808 10:07:35.695586 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.689782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:39733 (local address 127.0.0.1:54234)
0808 10:07:35.690163 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:35.690210 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:35.690294 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:35.690609 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:35.690612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:35.690618 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:35.690708 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:35.690712 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.692931 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.692932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:35.693742 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:35.693745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.695322 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.695323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:35.695398 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.695402 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:07:35.695457 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 10:07:35.695510 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":5,"thread_start_us":93,"threads_started":1}
I20260808 10:07:35.695658 3028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45db96349afb4cb9b36c6e2cef5b9d14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:35.695732 3251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.689941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35574 (local address 127.2.30.129:41095)
0808 10:07:35.690224 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:35.690227 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:35.690397 (+ 170us) server_negotiation.cc:407] Connection header received
0808 10:07:35.690442 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:35.690444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:35.690459 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:35.690528 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:35.690798 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.691951 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.694246 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.694976 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.695082 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.695143 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:35.695599 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 10:07:35.695647 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":86,"threads_started":1}
I20260808 10:07:35.695840 3250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.689925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54234 (local address 127.2.30.131:39733)
0808 10:07:35.690141 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:35.690147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:35.690281 (+ 134us) server_negotiation.cc:407] Connection header received
0808 10:07:35.690392 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:35.690394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:35.690414 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:35.690496 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:35.690815 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.691959 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.693849 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.695149 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.695216 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.695274 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:35.695698 (+ 424us) server_negotiation.cc:299] Negotiation successful
0808 10:07:35.695748 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":94,"threads_started":1}
I20260808 10:07:35.696359 2889 tablet_service.cc:1467] Processing CreateTablet for tablet 45db96349afb4cb9b36c6e2cef5b9d14 (DEFAULT_TABLE table=test-table [id=a94a2dcea18a4550acdfbe0be5c37b4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:35.696694 2889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45db96349afb4cb9b36c6e2cef5b9d14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:35.699301 3254 tablet_bootstrap.cc:492] T 45db96349afb4cb9b36c6e2cef5b9d14 P 8c3cb09400a54fb689f399164117fbf3: Bootstrap starting.
I20260808 10:07:35.700062 3254 tablet_bootstrap.cc:654] T 45db96349afb4cb9b36c6e2cef5b9d14 P 8c3cb09400a54fb689f399164117fbf3: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:35.700048 3255 tablet_bootstrap.cc:492] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1: Bootstrap starting.
I20260808 10:07:35.700115 3167 tablet_service.cc:1467] Processing CreateTablet for tablet 45db96349afb4cb9b36c6e2cef5b9d14 (DEFAULT_TABLE table=test-table [id=a94a2dcea18a4550acdfbe0be5c37b4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:35.700457 3167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45db96349afb4cb9b36c6e2cef5b9d14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:35.700665 3255 tablet_bootstrap.cc:654] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:35.700906 3254 log.cc:824] T 45db96349afb4cb9b36c6e2cef5b9d14 P 8c3cb09400a54fb689f399164117fbf3: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:35.701241 3255 log.cc:824] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:35.701900 3254 tablet_bootstrap.cc:492] T 45db96349afb4cb9b36c6e2cef5b9d14 P 8c3cb09400a54fb689f399164117fbf3: No bootstrap required, opened a new log
I20260808 10:07:35.701979 3254 ts_tablet_manager.cc:1397] T 45db96349afb4cb9b36c6e2cef5b9d14 P 8c3cb09400a54fb689f399164117fbf3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:07:35.702003 3255 tablet_bootstrap.cc:492] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1: No bootstrap required, opened a new log
I20260808 10:07:35.702060 3255 ts_tablet_manager.cc:1397] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:07:35.703872 3255 raft_consensus.cc:348] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46838f9dc44745f8988268d65c2e5a2a" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39733 } } peers { permanent_uuid: "8c3cb09400a54fb689f399164117fbf3" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38743 } } peers { permanent_uuid: "a2fba5b178c34ecaab25ac7b2d3727c1" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 41095 } }
I20260808 10:07:35.704061 3255 raft_consensus.cc:374] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:35.704111 3255 raft_consensus.cc:729] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fba5b178c34ecaab25ac7b2d3727c1, State: Initialized, Role: FOLLOWER
I20260808 10:07:35.704192 3258 tablet_bootstrap.cc:492] T 45db96349afb4cb9b36c6e2cef5b9d14 P 46838f9dc44745f8988268d65c2e5a2a: Bootstrap starting.
I20260808 10:07:35.704242 3255 consensus_queue.cc:260] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46838f9dc44745f8988268d65c2e5a2a" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39733 } } peers { permanent_uuid: "8c3cb09400a54fb689f399164117fbf3" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38743 } } peers { permanent_uuid: "a2fba5b178c34ecaab25ac7b2d3727c1" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 41095 } }
I20260808 10:07:35.704144 3254 raft_consensus.cc:348] T 45db96349afb4cb9b36c6e2cef5b9d14 P 8c3cb09400a54fb689f399164117fbf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46838f9dc44745f8988268d65c2e5a2a" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39733 } } peers { permanent_uuid: "8c3cb09400a54fb689f399164117fbf3" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38743 } } peers { permanent_uuid: "a2fba5b178c34ecaab25ac7b2d3727c1" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 41095 } }
I20260808 10:07:35.704353 3254 raft_consensus.cc:374] T 45db96349afb4cb9b36c6e2cef5b9d14 P 8c3cb09400a54fb689f399164117fbf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:35.704391 3254 raft_consensus.cc:729] T 45db96349afb4cb9b36c6e2cef5b9d14 P 8c3cb09400a54fb689f399164117fbf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3cb09400a54fb689f399164117fbf3, State: Initialized, Role: FOLLOWER
I20260808 10:07:35.704510 3254 consensus_queue.cc:260] T 45db96349afb4cb9b36c6e2cef5b9d14 P 8c3cb09400a54fb689f399164117fbf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46838f9dc44745f8988268d65c2e5a2a" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39733 } } peers { permanent_uuid: "8c3cb09400a54fb689f399164117fbf3" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38743 } } peers { permanent_uuid: "a2fba5b178c34ecaab25ac7b2d3727c1" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 41095 } }
I20260808 10:07:35.704550 3255 ts_tablet_manager.cc:1428] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 10:07:35.704814 3254 ts_tablet_manager.cc:1428] T 45db96349afb4cb9b36c6e2cef5b9d14 P 8c3cb09400a54fb689f399164117fbf3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:07:35.704854 2957 heartbeater.cc:503] Master 127.2.30.190:42165 was elected leader, sending a full tablet report...
I20260808 10:07:35.704887 3258 tablet_bootstrap.cc:654] T 45db96349afb4cb9b36c6e2cef5b9d14 P 46838f9dc44745f8988268d65c2e5a2a: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:35.705224 3097 heartbeater.cc:503] Master 127.2.30.190:42165 was elected leader, sending a full tablet report...
I20260808 10:07:35.705518 3258 log.cc:824] T 45db96349afb4cb9b36c6e2cef5b9d14 P 46838f9dc44745f8988268d65c2e5a2a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:35.706501 3258 tablet_bootstrap.cc:492] T 45db96349afb4cb9b36c6e2cef5b9d14 P 46838f9dc44745f8988268d65c2e5a2a: No bootstrap required, opened a new log
I20260808 10:07:35.706585 3258 ts_tablet_manager.cc:1397] T 45db96349afb4cb9b36c6e2cef5b9d14 P 46838f9dc44745f8988268d65c2e5a2a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:07:35.708007 3258 raft_consensus.cc:348] T 45db96349afb4cb9b36c6e2cef5b9d14 P 46838f9dc44745f8988268d65c2e5a2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46838f9dc44745f8988268d65c2e5a2a" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39733 } } peers { permanent_uuid: "8c3cb09400a54fb689f399164117fbf3" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38743 } } peers { permanent_uuid: "a2fba5b178c34ecaab25ac7b2d3727c1" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 41095 } }
I20260808 10:07:35.708170 3258 raft_consensus.cc:374] T 45db96349afb4cb9b36c6e2cef5b9d14 P 46838f9dc44745f8988268d65c2e5a2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:35.708217 3258 raft_consensus.cc:729] T 45db96349afb4cb9b36c6e2cef5b9d14 P 46838f9dc44745f8988268d65c2e5a2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46838f9dc44745f8988268d65c2e5a2a, State: Initialized, Role: FOLLOWER
I20260808 10:07:35.708345 3258 consensus_queue.cc:260] T 45db96349afb4cb9b36c6e2cef5b9d14 P 46838f9dc44745f8988268d65c2e5a2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46838f9dc44745f8988268d65c2e5a2a" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39733 } } peers { permanent_uuid: "8c3cb09400a54fb689f399164117fbf3" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38743 } } peers { permanent_uuid: "a2fba5b178c34ecaab25ac7b2d3727c1" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 41095 } }
I20260808 10:07:35.708613 3258 ts_tablet_manager.cc:1428] T 45db96349afb4cb9b36c6e2cef5b9d14 P 46838f9dc44745f8988268d65c2e5a2a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:07:35.708664 3237 heartbeater.cc:503] Master 127.2.30.190:42165 was elected leader, sending a full tablet report...
W20260808 10:07:35.744940 3098 tablet.cc:2369] T 45db96349afb4cb9b36c6e2cef5b9d14 P 8c3cb09400a54fb689f399164117fbf3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:07:35.820667 2958 tablet.cc:2369] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:07:35.890497 3259 raft_consensus.cc:482] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:07:35.890640 3259 raft_consensus.cc:504] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46838f9dc44745f8988268d65c2e5a2a" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39733 } } peers { permanent_uuid: "8c3cb09400a54fb689f399164117fbf3" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38743 } } peers { permanent_uuid: "a2fba5b178c34ecaab25ac7b2d3727c1" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 41095 } }
I20260808 10:07:35.891114 3259 leader_election.cc:288] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46838f9dc44745f8988268d65c2e5a2a (127.2.30.131:39733), 8c3cb09400a54fb689f399164117fbf3 (127.2.30.130:38743)
I20260808 10:07:35.895601 3250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.891665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:34259 (local address 127.2.30.131:39733)
0808 10:07:35.891722 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:35.891724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:35.891736 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:07:35.891775 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:35.891776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:35.891786 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:35.891897 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:35.892096 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.892977 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.894674 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.895144 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.895206 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.895213 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:35.895472 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 10:07:35.895524 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 10:07:35.895634 3263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.891426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:39733 (local address 127.2.30.129:34259)
0808 10:07:35.891556 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:35.891568 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:35.891627 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:35.891905 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:35.891906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:35.891907 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:35.892000 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:35.892003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.893088 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.893089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:35.894536 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:35.894541 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.895278 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.895280 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:35.895376 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.895381 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:07:35.895440 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 10:07:35.895479 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":63,"thread_start_us":81,"threads_started":1}
I20260808 10:07:35.895689 3253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.891791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:60149 (local address 127.2.30.130:38743)
0808 10:07:35.891868 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:35.891871 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:35.891880 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:07:35.891908 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:35.891909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:35.891922 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:35.892005 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:35.892149 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.892976 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.894642 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.895239 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.895332 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.895342 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:35.895544 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 10:07:35.895597 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:07:35.895663 3264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.891625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:38743 (local address 127.2.30.129:60149)
0808 10:07:35.891725 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:35.891735 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:35.891775 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:35.892012 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:35.892012 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:35.892014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:35.892074 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:35.892076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.893088 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.893089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:35.894537 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:35.894541 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.895345 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.895346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:35.895448 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.895453 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:07:35.895514 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 10:07:35.895555 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":101,"thread_start_us":64,"threads_started":1}
I20260808 10:07:35.896129 3188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45db96349afb4cb9b36c6e2cef5b9d14" candidate_uuid: "a2fba5b178c34ecaab25ac7b2d3727c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46838f9dc44745f8988268d65c2e5a2a" is_pre_election: true
I20260808 10:07:35.896308 3188 raft_consensus.cc:2454] T 45db96349afb4cb9b36c6e2cef5b9d14 P 46838f9dc44745f8988268d65c2e5a2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fba5b178c34ecaab25ac7b2d3727c1 in term 0.
I20260808 10:07:35.896337 3048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45db96349afb4cb9b36c6e2cef5b9d14" candidate_uuid: "a2fba5b178c34ecaab25ac7b2d3727c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3cb09400a54fb689f399164117fbf3" is_pre_election: true
I20260808 10:07:35.896526 3048 raft_consensus.cc:2454] T 45db96349afb4cb9b36c6e2cef5b9d14 P 8c3cb09400a54fb689f399164117fbf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fba5b178c34ecaab25ac7b2d3727c1 in term 0.
I20260808 10:07:35.896625 2841 leader_election.cc:302] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46838f9dc44745f8988268d65c2e5a2a, a2fba5b178c34ecaab25ac7b2d3727c1; no voters:
I20260808 10:07:35.896766 3259 raft_consensus.cc:2781] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:07:35.896835 3259 raft_consensus.cc:482] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:07:35.896863 3259 raft_consensus.cc:3037] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:35.897534 3259 raft_consensus.cc:504] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46838f9dc44745f8988268d65c2e5a2a" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39733 } } peers { permanent_uuid: "8c3cb09400a54fb689f399164117fbf3" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38743 } } peers { permanent_uuid: "a2fba5b178c34ecaab25ac7b2d3727c1" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 41095 } }
I20260808 10:07:35.897676 3259 leader_election.cc:288] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [CANDIDATE]: Term 1 election: Requested vote from peers 46838f9dc44745f8988268d65c2e5a2a (127.2.30.131:39733), 8c3cb09400a54fb689f399164117fbf3 (127.2.30.130:38743)
I20260808 10:07:35.897882 3188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45db96349afb4cb9b36c6e2cef5b9d14" candidate_uuid: "a2fba5b178c34ecaab25ac7b2d3727c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46838f9dc44745f8988268d65c2e5a2a"
I20260808 10:07:35.897886 3048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45db96349afb4cb9b36c6e2cef5b9d14" candidate_uuid: "a2fba5b178c34ecaab25ac7b2d3727c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3cb09400a54fb689f399164117fbf3"
I20260808 10:07:35.897966 3188 raft_consensus.cc:3037] T 45db96349afb4cb9b36c6e2cef5b9d14 P 46838f9dc44745f8988268d65c2e5a2a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:35.898008 3048 raft_consensus.cc:3037] T 45db96349afb4cb9b36c6e2cef5b9d14 P 8c3cb09400a54fb689f399164117fbf3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:35.898749 3188 raft_consensus.cc:2454] T 45db96349afb4cb9b36c6e2cef5b9d14 P 46838f9dc44745f8988268d65c2e5a2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fba5b178c34ecaab25ac7b2d3727c1 in term 1.
I20260808 10:07:35.898877 3048 raft_consensus.cc:2454] T 45db96349afb4cb9b36c6e2cef5b9d14 P 8c3cb09400a54fb689f399164117fbf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fba5b178c34ecaab25ac7b2d3727c1 in term 1.
I20260808 10:07:35.898926 2841 leader_election.cc:302] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46838f9dc44745f8988268d65c2e5a2a, a2fba5b178c34ecaab25ac7b2d3727c1; no voters:
I20260808 10:07:35.899065 3259 raft_consensus.cc:2781] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:35.899255 3259 raft_consensus.cc:686] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [term 1 LEADER]: Becoming Leader. State: Replica: a2fba5b178c34ecaab25ac7b2d3727c1, State: Running, Role: LEADER
I20260808 10:07:35.899364 3259 consensus_queue.cc:237] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46838f9dc44745f8988268d65c2e5a2a" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39733 } } peers { permanent_uuid: "8c3cb09400a54fb689f399164117fbf3" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38743 } } peers { permanent_uuid: "a2fba5b178c34ecaab25ac7b2d3727c1" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 41095 } }
I20260808 10:07:35.900293 2757 catalog_manager.cc:5495] T 45db96349afb4cb9b36c6e2cef5b9d14 P a2fba5b178c34ecaab25ac7b2d3727c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fba5b178c34ecaab25ac7b2d3727c1 (127.2.30.129). New cstate: current_term: 1 leader_uuid: "a2fba5b178c34ecaab25ac7b2d3727c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46838f9dc44745f8988268d65c2e5a2a" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3cb09400a54fb689f399164117fbf3" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2fba5b178c34ecaab25ac7b2d3727c1" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 41095 } health_report { overall_health: HEALTHY } } }
W20260808 10:07:35.913249 3238 tablet.cc:2369] T 45db96349afb4cb9b36c6e2cef5b9d14 P 46838f9dc44745f8988268d65c2e5a2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 10:07:35 dist-test-slave-w206 krb5kdc[2710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.129@KRBTEST.COM
I20260808 10:07:35.926860 3251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.918280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35576 (local address 127.2.30.129:41095)
0808 10:07:35.918351 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:35.918354 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:35.918553 (+ 199us) server_negotiation.cc:407] Connection header received
0808 10:07:35.918586 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:35.918587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:35.918600 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:35.918662 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:35.919311 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.919820 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.920555 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.920743 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.920799 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.923438 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:35.923462 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:35.923463 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:35.923471 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:35.925152 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:35.925671 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:35.925679 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:35.925680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:35.925770 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:35.926017 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:35.926019 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:35.926019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:35.926286 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:35.926487 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:35.926587 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 10:07:35.926701 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 10:07:35 dist-test-slave-w206 krb5kdc[2710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183655, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:07:35 dist-test-slave-w206 krb5kdc[2710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183655, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.30.129@KRBTEST.COM
I20260808 10:07:35.953254 3251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.945339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35580 (local address 127.2.30.129:41095)
0808 10:07:35.945451 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:35.945453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:35.945464 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:07:35.945519 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:35.945520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:35.945530 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:35.945590 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:35.946249 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.946756 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.947595 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.947815 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.947883 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.950319 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:35.950321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:35.950322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:35.950330 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:35.951750 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:35.952178 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:35.952179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:35.952179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:35.952245 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:35.952484 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:35.952485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:35.952485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:35.952611 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:35.952749 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:35.952962 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 10:07:35.953068 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
Aug 08 10:07:35 dist-test-slave-w206 krb5kdc[2710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:07:35 dist-test-slave-w206 krb5kdc[2710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.129@KRBTEST.COM
I20260808 10:07:35.979885 3251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:35.971725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35596 (local address 127.2.30.129:41095)
0808 10:07:35.971807 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:35.971810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:35.972010 (+ 200us) server_negotiation.cc:407] Connection header received
0808 10:07:35.972045 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:35.972047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:35.972058 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:35.972126 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:35.972737 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.973212 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.974201 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:35.974410 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:35.974475 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:35.977030 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:35.977031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:35.977031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:35.977038 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:35.978632 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:35.978962 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:35.978964 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:35.978964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:35.979102 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:35.979325 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:35.979326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:35.979326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:35.979453 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:35.979571 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:35.979634 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 10:07:35.979713 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 10:07:35.980602 2869 tablet_service.cc:2908] Rejecting scan request for tablet 45db96349afb4cb9b36c6e2cef5b9d14: Uninitialized: clean time has not yet been initialized
I20260808 10:07:35.981676 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 2823
I20260808 10:07:35.988346 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 2964
I20260808 10:07:35.994048 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 3102
I20260808 10:07:36.002758 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 2723
2026-08-08T10:07:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2272 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:36 dist-test-slave-w206 krb5kdc[3279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:36 dist-test-slave-w206 krb5kdc[3279](info): set up 2 sockets
Aug 08 10:07:36 dist-test-slave-w206 krb5kdc[3279](info): commencing operation
krb5kdc: starting...
W20260808 10:07:37.113621 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:07:37 dist-test-slave-w206 krb5kdc[3279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:07:37.245613 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:46551
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:46551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:37.336035 3292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:37.336321 3292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:37.336365 3292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:37.337626 3292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:37.337709 3292 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 10:07:37.337734 3292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:37.337769 3292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:37.337819 3292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:37.337850 3292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:37.339689 3292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:46551
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:46551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:37.340023 3292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:37.340282 3292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:37.346036 3301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:37.346099 3292 server_base.cc:1034] running on GCE node
W20260808 10:07:37.346050 3297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:37.346062 3298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:37.346582 3292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:37.346907 3292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:37.348059 3292 hybrid_clock.cc:648] HybridClock initialized: now 1786183657348032 us; error 38 us; skew 500 ppm
Aug 08 10:07:37 dist-test-slave-w206 krb5kdc[3279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183657, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:37.355705 3292 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:07:37.356175 3292 webserver.cc:460] Webserver started at http://127.2.30.190:46249/ using document root <none> and password file <none>
I20260808 10:07:37.356385 3292 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:37.356428 3292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:37.356521 3292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:37.357622 3292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "6baa28ca53a6419ea32a731e0b4d79b1"
format_stamp: "Formatted at 2026-08-08 10:07:37 on dist-test-slave-w206"
server_key: "d3262f699d49897b5b6d2432c01179e9"
server_key_iv: "5fcc3623ead2445f1409855e1c38c155"
server_key_version: "encryptionkey@0"
I20260808 10:07:37.357957 3292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "6baa28ca53a6419ea32a731e0b4d79b1"
format_stamp: "Formatted at 2026-08-08 10:07:37 on dist-test-slave-w206"
server_key: "d3262f699d49897b5b6d2432c01179e9"
server_key_iv: "5fcc3623ead2445f1409855e1c38c155"
server_key_version: "encryptionkey@0"
I20260808 10:07:37.359411 3292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:07:37.360282 3308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:37.360538 3292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:07:37.360637 3292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "6baa28ca53a6419ea32a731e0b4d79b1"
format_stamp: "Formatted at 2026-08-08 10:07:37 on dist-test-slave-w206"
server_key: "d3262f699d49897b5b6d2432c01179e9"
server_key_iv: "5fcc3623ead2445f1409855e1c38c155"
server_key_version: "encryptionkey@0"
I20260808 10:07:37.360704 3292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:37.376564 3292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:37.379091 3292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:37.379215 3292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:37.384238 3292 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:46551
I20260808 10:07:37.384359 3372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:46551 every 8 connection(s)
I20260808 10:07:37.384685 3292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:37.385481 3373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:37.387949 3373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1: Bootstrap starting.
I20260808 10:07:37.388576 3373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:37.388885 3373 log.cc:824] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:37.389652 3373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1: No bootstrap required, opened a new log
I20260808 10:07:37.392274 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 3292
I20260808 10:07:37.392198 3373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6baa28ca53a6419ea32a731e0b4d79b1" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46551 } }
I20260808 10:07:37.392338 3373 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:37.392385 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:07:37.392360 3373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6baa28ca53a6419ea32a731e0b4d79b1, State: Initialized, Role: FOLLOWER
I20260808 10:07:37.392529 3373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6baa28ca53a6419ea32a731e0b4d79b1" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46551 } }
I20260808 10:07:37.392580 2170 external_mini_cluster.cc:1428] Setting key f90c0543b763a35171470e18ea3b53c3
I20260808 10:07:37.392607 3373 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:37.392663 3373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:37.392711 3373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:37.393501 3373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6baa28ca53a6419ea32a731e0b4d79b1" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46551 } }
I20260808 10:07:37.393630 3373 leader_election.cc:302] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6baa28ca53a6419ea32a731e0b4d79b1; no voters:
I20260808 10:07:37.393833 3373 leader_election.cc:288] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:37.393877 3378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:37.393988 3378 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [term 1 LEADER]: Becoming Leader. State: Replica: 6baa28ca53a6419ea32a731e0b4d79b1, State: Running, Role: LEADER
I20260808 10:07:37.394073 3378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6baa28ca53a6419ea32a731e0b4d79b1" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46551 } }
I20260808 10:07:37.394107 3373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:37.395238 3380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6baa28ca53a6419ea32a731e0b4d79b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6baa28ca53a6419ea32a731e0b4d79b1" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46551 } } }
I20260808 10:07:37.395336 3380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:37.395612 3379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6baa28ca53a6419ea32a731e0b4d79b1. Latest consensus state: current_term: 1 leader_uuid: "6baa28ca53a6419ea32a731e0b4d79b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6baa28ca53a6419ea32a731e0b4d79b1" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46551 } } }
I20260808 10:07:37.395741 3379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:37.395954 3387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:37.396721 3387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:37.398276 3387 catalog_manager.cc:1347] Generated new cluster ID: fa7eb16b2e78405dbc022ac11469dcae
I20260808 10:07:37.398388 3387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:07:37 dist-test-slave-w206 krb5kdc[3279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:37.405822 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.393112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60724 (local address 127.2.30.190:46551)
0808 10:07:37.393309 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:37.393317 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:37.393373 (+ 56us) server_negotiation.cc:407] Connection header received
0808 10:07:37.393471 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:37.393477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:37.393520 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:37.393679 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:37.394510 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.396522 (+ 2012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.397587 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.398234 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.398351 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.401254 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:37.401259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:37.401259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:37.401273 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:37.404056 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:37.404484 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:37.404488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:37.404489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:37.404617 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:37.404829 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:37.404831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:37.404831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:37.405179 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:37.405360 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:37.405437 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 10:07:37.405591 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
I20260808 10:07:37.415280 3387 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:37.415817 3387 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:37.420604 3387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1: Generated new TSK 0
I20260808 10:07:37.420812 3387 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:07:37.450531 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46551
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:37.536267 3399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:37.536491 3399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:37.536520 3399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:37.537798 3399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:37.537853 3399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:37.537874 3399 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:07:37.537894 3399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:07:37.539579 3399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.30.190:46551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:37.539871 3399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:37.540165 3399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:37.545529 3407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:37.545549 3405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:37.545614 3404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:37.545802 3399 server_base.cc:1034] running on GCE node
I20260808 10:07:37.546051 3399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:37.546339 3399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:37.547506 3399 hybrid_clock.cc:648] HybridClock initialized: now 1786183657547508 us; error 45 us; skew 500 ppm
Aug 08 10:07:37 dist-test-slave-w206 krb5kdc[3279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183657, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:37.550269 3399 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:07:37.550771 3399 webserver.cc:460] Webserver started at http://127.2.30.129:39839/ using document root <none> and password file <none>
I20260808 10:07:37.551080 3399 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:37.551152 3399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:37.551324 3399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:37.552428 3399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "3bcb3b5c2b6d41058729a11014eb16cd"
format_stamp: "Formatted at 2026-08-08 10:07:37 on dist-test-slave-w206"
server_key: "76941e003974c0b25970c7ed39e8dca9"
server_key_iv: "5844e5676699330eadd57e77cf41c0c8"
server_key_version: "encryptionkey@0"
I20260808 10:07:37.552821 3399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "3bcb3b5c2b6d41058729a11014eb16cd"
format_stamp: "Formatted at 2026-08-08 10:07:37 on dist-test-slave-w206"
server_key: "76941e003974c0b25970c7ed39e8dca9"
server_key_iv: "5844e5676699330eadd57e77cf41c0c8"
server_key_version: "encryptionkey@0"
I20260808 10:07:37.554394 3399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:07:37.555339 3415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:37.555516 3399 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:37.555632 3399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "3bcb3b5c2b6d41058729a11014eb16cd"
format_stamp: "Formatted at 2026-08-08 10:07:37 on dist-test-slave-w206"
server_key: "76941e003974c0b25970c7ed39e8dca9"
server_key_iv: "5844e5676699330eadd57e77cf41c0c8"
server_key_version: "encryptionkey@0"
I20260808 10:07:37.555709 3399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:37.571142 3399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:37.573107 3399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:37.573302 3399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:37.573860 3399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:37.573926 3399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:37.573962 3399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:37.573993 3399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:37.581297 3399 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:39907
Aug 08 10:07:37 dist-test-slave-w206 krb5kdc[3279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183657, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:37.581383 3535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:39907 every 8 connection(s)
I20260808 10:07:37.581727 3399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:07:37.584920 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.574552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:45221 (local address 127.2.30.190:46551)
0808 10:07:37.574627 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:37.574630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:37.574849 (+ 219us) server_negotiation.cc:407] Connection header received
0808 10:07:37.575469 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:37.575471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:37.575486 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:37.575572 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:37.576473 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.577355 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.579032 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.579368 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.579452 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.582013 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:37.582015 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:37.582016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:37.582028 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:37.583627 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:37.584040 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:37.584040 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:37.584041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:37.584102 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:37.584326 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:37.584326 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:37.584327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:37.584455 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:37.584519 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:37.584782 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 10:07:37.584855 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:07:37.585036 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.574477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46551 (local address 127.2.30.129:45221)
0808 10:07:37.574702 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:37.574746 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:37.575354 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:37.575658 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:37.575663 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:37.576067 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:37.576372 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.576381 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.577492 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.577494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.578900 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.578903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.579459 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.579460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.579574 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.580196 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:37.580206 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:37.581878 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:37.583736 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:37.583743 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:37.583745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:37.583968 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:37.584164 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:37.584164 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:37.584164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:37.584267 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:37.584512 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:37.584518 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:37.584636 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 10:07:37.584745 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 10:07:37.584874 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":128,"threads_started":1}
I20260808 10:07:37.585669 3536 heartbeater.cc:347] Connected to a master server at 127.2.30.190:46551
I20260808 10:07:37.585755 3536 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:37.585955 3536 heartbeater.cc:511] Master 127.2.30.190:46551 requested a full tablet report, sending...
I20260808 10:07:37.586602 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 3399
I20260808 10:07:37.586637 3325 ts_manager.cc:194] Registered new tserver with Master: 3bcb3b5c2b6d41058729a11014eb16cd (127.2.30.129:39907)
I20260808 10:07:37.586706 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:07:37.586926 2170 external_mini_cluster.cc:1428] Setting key 5cbe342a135eea98735aedc713c2f683
I20260808 10:07:37.587608 3325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:45221
I20260808 10:07:37.592962 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.587023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:48349 (local address 127.2.30.190:46551)
0808 10:07:37.587074 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:37.587077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:37.587213 (+ 136us) server_negotiation.cc:407] Connection header received
0808 10:07:37.587315 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:37.587316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:37.587328 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:37.587380 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:37.587740 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.588329 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.589047 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.589261 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.589336 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.590296 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:37.590297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:37.590298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:37.590305 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:37.591727 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:37.592135 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:37.592136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:37.592136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:37.592200 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:37.592419 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:37.592420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:37.592421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:37.592536 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:37.592608 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:37.592785 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 10:07:37.592868 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:07:37.592969 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.586929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46551 (local address 127.2.30.129:48349)
0808 10:07:37.587155 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:37.587165 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:37.587236 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:37.587385 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:37.587386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:37.587601 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:37.587651 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.587653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.588429 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.588430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.588955 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.588957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.589334 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.589335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.589411 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.589892 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:37.589897 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:37.590189 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:37.591827 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:37.591828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:37.591829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:37.592049 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:37.592279 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:37.592280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:37.592280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:37.592347 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:37.592611 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:37.592612 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:37.592658 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 10:07:37.592756 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 10:07:37.592868 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:07:37.617309 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46551
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:37.702781 3545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:37.703024 3545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:37.703051 3545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:37.704332 3545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:37.704380 3545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:37.704401 3545 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:07:37.704420 3545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:07:37.705929 3545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.30.190:46551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:37.706187 3545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:37.706457 3545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:37.711756 3554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:37.711782 3552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:37.711756 3551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:37.711954 3545 server_base.cc:1034] running on GCE node
I20260808 10:07:37.712361 3545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:37.712630 3545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:37.713811 3545 hybrid_clock.cc:648] HybridClock initialized: now 1786183657713798 us; error 32 us; skew 500 ppm
Aug 08 10:07:37 dist-test-slave-w206 krb5kdc[3279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183657, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:37.716533 3545 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:07:37.716961 3545 webserver.cc:460] Webserver started at http://127.2.30.130:32895/ using document root <none> and password file <none>
I20260808 10:07:37.717175 3545 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:37.717217 3545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:37.717308 3545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:37.718401 3545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "db01a79db2754211a161c93f5cee35e9"
format_stamp: "Formatted at 2026-08-08 10:07:37 on dist-test-slave-w206"
server_key: "82e9289dd439718fe4716a5544c62265"
server_key_iv: "cd1ed4e60d63ed538626b94b9bee32f8"
server_key_version: "encryptionkey@0"
I20260808 10:07:37.718709 3545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "db01a79db2754211a161c93f5cee35e9"
format_stamp: "Formatted at 2026-08-08 10:07:37 on dist-test-slave-w206"
server_key: "82e9289dd439718fe4716a5544c62265"
server_key_iv: "cd1ed4e60d63ed538626b94b9bee32f8"
server_key_version: "encryptionkey@0"
I20260808 10:07:37.720192 3545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 10:07:37.720970 3561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:37.721127 3545 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:07:37.721212 3545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "db01a79db2754211a161c93f5cee35e9"
format_stamp: "Formatted at 2026-08-08 10:07:37 on dist-test-slave-w206"
server_key: "82e9289dd439718fe4716a5544c62265"
server_key_iv: "cd1ed4e60d63ed538626b94b9bee32f8"
server_key_version: "encryptionkey@0"
I20260808 10:07:37.721316 3545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:37.752445 3545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:37.755095 3545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:37.755223 3545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:37.755738 3545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:37.755779 3545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:37.755802 3545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:37.755823 3545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:37.762854 3545 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:45869
I20260808 10:07:37.762954 3680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:45869 every 8 connection(s)
I20260808 10:07:37.763363 3545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:07:37.763679 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 3545
Aug 08 10:07:37 dist-test-slave-w206 krb5kdc[3279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183657, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:37.763767 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:07:37.763968 2170 external_mini_cluster.cc:1428] Setting key a8c302b7fe135ba5ce5b407f6eec084f
I20260808 10:07:37.767692 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.756360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:55073 (local address 127.2.30.190:46551)
0808 10:07:37.756424 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:37.756427 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:37.756670 (+ 243us) server_negotiation.cc:407] Connection header received
0808 10:07:37.757303 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:37.757305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:37.757319 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:37.757397 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:37.758349 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.759302 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.760963 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.761298 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.761383 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.764417 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:37.764419 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:37.764419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:37.764425 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:37.766153 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:37.766590 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:37.766591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:37.766592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:37.766666 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:37.767021 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:37.767022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:37.767022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:37.767154 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:37.767209 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:37.767558 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 10:07:37.767626 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 10:07:37.767982 3570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.756357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46551 (local address 127.2.30.130:55073)
0808 10:07:37.756524 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:37.756561 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:37.757166 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:37.757487 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:37.757492 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:37.757894 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:37.758187 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.758193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.759406 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.759407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.760859 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.760867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.761423 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.761425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.761586 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.762294 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:37.762305 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:37.764322 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:37.766240 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:37.766246 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:37.766248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:37.766479 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:37.766746 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:37.766747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:37.766748 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:37.766910 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:37.767223 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:37.767229 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:37.767408 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 10:07:37.767535 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 10:07:37.767726 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260808 10:07:37.768522 3681 heartbeater.cc:347] Connected to a master server at 127.2.30.190:46551
I20260808 10:07:37.768648 3681 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:37.768886 3681 heartbeater.cc:511] Master 127.2.30.190:46551 requested a full tablet report, sending...
I20260808 10:07:37.769302 3326 ts_manager.cc:194] Registered new tserver with Master: db01a79db2754211a161c93f5cee35e9 (127.2.30.130:45869)
I20260808 10:07:37.769846 3326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:55073
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
I20260808 10:07:37.777666 3688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.769652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46551 (local address 127.2.30.130:59227)
0808 10:07:37.771094 (+ 1442us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:37.771108 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:37.771184 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:37.771340 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:37.771342 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:37.771585 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:37.771664 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.771666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.772350 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.772352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.773229 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.773232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.773603 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.773605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.773688 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.774229 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:37.774234 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:37.774653 (+ 419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:37.776568 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:37.776569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:37.776569 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:37.776739 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:37.776933 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:37.776934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:37.776934 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:37.776989 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:37.777234 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:37.777235 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:37.777283 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 10:07:37.777390 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 10:07:37.777514 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1408,"thread_start_us":85,"threads_started":1}
I20260808 10:07:37.777755 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.769721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:59227 (local address 127.2.30.190:46551)
0808 10:07:37.769757 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:37.769759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:37.771181 (+ 1422us) server_negotiation.cc:407] Connection header received
0808 10:07:37.771250 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:37.771251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:37.771260 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:37.771310 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:37.771771 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.772236 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.773337 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.773517 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.773576 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.774741 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:37.774742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:37.774743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:37.774749 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:37.776484 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:37.776812 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:37.776813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:37.776814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:37.776871 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:37.777038 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:37.777038 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:37.777038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:37.777178 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:37.777228 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:37.777428 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 10:07:37.777633 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:07:37.795293 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46551
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:37.881381 3690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:37.881606 3690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:37.881633 3690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:37.882877 3690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:37.882926 3690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:37.882948 3690 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:07:37.882966 3690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:07:37.884542 3690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.30.190:46551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:37.884802 3690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:37.885056 3690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:37.890584 3700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:37.890580 3697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:37.890609 3698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:37.890938 3690 server_base.cc:1034] running on GCE node
I20260808 10:07:37.891173 3690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:37.891512 3690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:37.892680 3690 hybrid_clock.cc:648] HybridClock initialized: now 1786183657892647 us; error 51 us; skew 500 ppm
Aug 08 10:07:37 dist-test-slave-w206 krb5kdc[3279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183657, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:37.895656 3690 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:07:37.896270 3690 webserver.cc:460] Webserver started at http://127.2.30.131:35737/ using document root <none> and password file <none>
I20260808 10:07:37.896521 3690 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:37.896591 3690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:37.896754 3690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:37.897866 3690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "c45f107da5df4ccc9a213e419d2f2c04"
format_stamp: "Formatted at 2026-08-08 10:07:37 on dist-test-slave-w206"
server_key: "b3c97f25787ab5928271494e2718b7eb"
server_key_iv: "1ac8518e50cd90c9547c4b210140fcb7"
server_key_version: "encryptionkey@0"
I20260808 10:07:37.898247 3690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "c45f107da5df4ccc9a213e419d2f2c04"
format_stamp: "Formatted at 2026-08-08 10:07:37 on dist-test-slave-w206"
server_key: "b3c97f25787ab5928271494e2718b7eb"
server_key_iv: "1ac8518e50cd90c9547c4b210140fcb7"
server_key_version: "encryptionkey@0"
I20260808 10:07:37.899837 3690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:07:37.900780 3707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:37.901049 3690 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:37.901172 3690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "c45f107da5df4ccc9a213e419d2f2c04"
format_stamp: "Formatted at 2026-08-08 10:07:37 on dist-test-slave-w206"
server_key: "b3c97f25787ab5928271494e2718b7eb"
server_key_iv: "1ac8518e50cd90c9547c4b210140fcb7"
server_key_version: "encryptionkey@0"
I20260808 10:07:37.901297 3690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:37.921218 3690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:37.923611 3690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:37.923800 3690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:37.924562 3690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:37.924618 3690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:37.924649 3690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:37.924671 3690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:07:37 dist-test-slave-w206 krb5kdc[3279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183657, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:37.932891 3690 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:45015
I20260808 10:07:37.932924 3824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:45015 every 8 connection(s)
I20260808 10:07:37.933349 3690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:07:37.935559 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.924983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:55223 (local address 127.2.30.190:46551)
0808 10:07:37.925030 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:37.925032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:37.925280 (+ 248us) server_negotiation.cc:407] Connection header received
0808 10:07:37.925923 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:37.925925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:37.925939 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:37.925999 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:37.926916 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.927491 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.928531 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.928792 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.928885 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.931484 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:37.931485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:37.931486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:37.931493 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:37.933234 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:37.934346 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:37.934348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:37.934348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:37.934445 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:37.934762 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:37.934763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:37.934763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:37.934905 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:37.935019 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:37.935368 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 10:07:37.935465 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:07:37.935739 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.924967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46551 (local address 127.2.30.131:55223)
0808 10:07:37.925145 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:37.925187 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:37.925807 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:37.926071 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:37.926075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:37.926505 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:37.926823 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.926830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.927600 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.927601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.928433 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.928436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.928898 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.928900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.929037 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.929602 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:37.929612 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:37.931389 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:37.933329 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:37.933336 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:37.933339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:37.933646 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:37.934523 (+ 877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:37.934524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:37.934524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:37.934672 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:37.935014 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:37.935020 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:37.935188 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 10:07:37.935324 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 10:07:37.935497 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260808 10:07:37.936352 3825 heartbeater.cc:347] Connected to a master server at 127.2.30.190:46551
I20260808 10:07:37.936474 3825 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:37.936730 3825 heartbeater.cc:511] Master 127.2.30.190:46551 requested a full tablet report, sending...
I20260808 10:07:37.937177 3325 ts_manager.cc:194] Registered new tserver with Master: c45f107da5df4ccc9a213e419d2f2c04 (127.2.30.131:45015)
I20260808 10:07:37.937707 3325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:55223
I20260808 10:07:37.941594 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 3690
I20260808 10:07:37.941689 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:07:37.941875 2170 external_mini_cluster.cc:1428] Setting key 99e3550f52509fb8a85b63640d329dc1
I20260808 10:07:37.942742 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:07:37.943611 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.937544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:57157 (local address 127.2.30.190:46551)
0808 10:07:37.937586 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:37.937588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:37.937654 (+ 66us) server_negotiation.cc:407] Connection header received
0808 10:07:37.937722 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:37.937723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:37.937733 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:37.937796 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:37.938208 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.938800 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.939582 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.939802 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.939860 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.940812 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:37.940813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:37.940814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:37.940820 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:37.942389 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:37.942747 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:37.942767 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:37.942768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:37.942851 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:37.943074 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:37.943075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:37.943076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:37.943193 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:37.943271 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:37.943436 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 10:07:37.943515 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:07:37.943617 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.937473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46551 (local address 127.2.30.131:57157)
0808 10:07:37.937592 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:37.937603 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:37.937665 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:37.937825 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:37.937826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:37.938055 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:37.938125 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.938127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.938901 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.938901 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.939510 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.939512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.939873 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.939874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.939940 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.940437 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:37.940441 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:37.940729 (+ 288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:37.942484 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:37.942485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:37.942485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:37.942663 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:37.942912 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:37.942913 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:37.942913 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:37.943021 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:37.943267 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:37.943268 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:37.943317 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 10:07:37.943409 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 10:07:37.943517 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":68,"threads_started":1}
Aug 08 10:07:37 dist-test-slave-w206 krb5kdc[3279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183657, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:07:37 dist-test-slave-w206 krb5kdc[3279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183657, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:37.968371 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.960725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60738 (local address 127.2.30.190:46551)
0808 10:07:37.960800 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:37.960802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:37.960810 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:07:37.960840 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:37.960841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:37.960850 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:37.960917 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:37.961511 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.962006 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.962762 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.962960 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.963114 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.965526 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:37.965527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:37.965528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:37.965536 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:37.967077 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:37.967500 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:37.967502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:37.967502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:37.967569 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:37.967779 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:37.967780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:37.967781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:37.967915 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:37.968024 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:37.968169 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 10:07:37.968259 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:07:37.970027 3325 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:07:37.971868 3325 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:07:37.978938 3845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.974395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:45869 (local address 127.0.0.1:39974)
0808 10:07:37.974587 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:37.974631 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:37.974750 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:37.975387 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:37.975390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:37.975393 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:37.975470 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.975472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.976507 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.976508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.977581 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.977584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.978569 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.978571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.978669 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.978686 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 10:07:37.978762 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:07:37.978821 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":61,"threads_started":1}
I20260808 10:07:37.979229 3844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.974400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:45015 (local address 127.0.0.1:39276)
0808 10:07:37.974652 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:37.974664 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:37.974767 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:37.975775 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:37.975776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:37.975778 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:37.975816 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.975817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.977075 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.977076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.978014 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.978016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.978784 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.978786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.978860 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.978864 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:07:37.978932 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 10:07:37.979018 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":111,"threads_started":1}
I20260808 10:07:37.979383 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.974498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39276 (local address 127.2.30.131:45015)
0808 10:07:37.975576 (+ 1078us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:37.975580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:37.975590 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:07:37.975628 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:37.975630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:37.975647 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:37.975761 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:37.975904 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.976975 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.978099 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.978691 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.978781 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.978838 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:37.978958 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 10:07:37.979057 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1032,"thread_start_us":96,"threads_started":1}
I20260808 10:07:37.979414 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.974668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49524 (local address 127.2.30.129:39907)
0808 10:07:37.975228 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:37.975232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:37.975243 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:07:37.975286 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:37.975287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:37.975299 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:37.975434 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:37.975591 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.976786 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.978117 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.978886 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.978948 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.979021 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:37.979216 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 10:07:37.979262 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":179,"threads_started":1}
I20260808 10:07:37.979560 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.974927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39974 (local address 127.2.30.130:45869)
0808 10:07:37.975185 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:37.975190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:37.975200 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:07:37.975245 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:37.975247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:37.975265 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:37.975356 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:37.975552 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.976418 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.977677 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.978474 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.978552 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.978611 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:37.979415 (+ 804us) server_negotiation.cc:299] Negotiation successful
0808 10:07:37.979465 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":162,"threads_started":1}
I20260808 10:07:37.979650 3843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.974408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:39907 (local address 127.0.0.1:49524)
0808 10:07:37.974843 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:37.974853 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:37.974918 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:37.975444 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:37.975445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:37.975447 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:37.975506 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.975524 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.976953 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.976954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.978012 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.978015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.979053 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.979054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.979112 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.979117 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:07:37.979525 (+ 408us) client_negotiation.cc:240] Negotiation successful
0808 10:07:37.979570 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":37,"thread_start_us":97,"threads_started":1}
I20260808 10:07:37.980100 3758 tablet_service.cc:1467] Processing CreateTablet for tablet 159d958ceae444478dda5026fefa6d32 (DEFAULT_TABLE table=test-table [id=8f5607e3d0f04c3699b06e6d1bdc4bc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:37.980120 3614 tablet_service.cc:1467] Processing CreateTablet for tablet 159d958ceae444478dda5026fefa6d32 (DEFAULT_TABLE table=test-table [id=8f5607e3d0f04c3699b06e6d1bdc4bc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:37.980463 3758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 159d958ceae444478dda5026fefa6d32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:37.980474 3614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 159d958ceae444478dda5026fefa6d32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:37.983042 3849 tablet_bootstrap.cc:492] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04: Bootstrap starting.
I20260808 10:07:37.983805 3850 tablet_bootstrap.cc:492] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9: Bootstrap starting.
I20260808 10:07:37.983700 3467 tablet_service.cc:1467] Processing CreateTablet for tablet 159d958ceae444478dda5026fefa6d32 (DEFAULT_TABLE table=test-table [id=8f5607e3d0f04c3699b06e6d1bdc4bc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:37.984015 3849 tablet_bootstrap.cc:654] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:37.984078 3467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 159d958ceae444478dda5026fefa6d32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:37.984370 3849 log.cc:824] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:37.984454 3850 tablet_bootstrap.cc:654] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:37.984798 3850 log.cc:824] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:37.985212 3849 tablet_bootstrap.cc:492] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04: No bootstrap required, opened a new log
I20260808 10:07:37.985301 3849 ts_tablet_manager.cc:1397] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:07:37.985594 3850 tablet_bootstrap.cc:492] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9: No bootstrap required, opened a new log
I20260808 10:07:37.985674 3850 ts_tablet_manager.cc:1397] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:07:37.987561 3850 raft_consensus.cc:348] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } }
I20260808 10:07:37.987721 3850 raft_consensus.cc:374] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:37.987607 3849 raft_consensus.cc:348] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } }
I20260808 10:07:37.987767 3850 raft_consensus.cc:729] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db01a79db2754211a161c93f5cee35e9, State: Initialized, Role: FOLLOWER
I20260808 10:07:37.987772 3849 raft_consensus.cc:374] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:37.987813 3849 raft_consensus.cc:729] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c45f107da5df4ccc9a213e419d2f2c04, State: Initialized, Role: FOLLOWER
I20260808 10:07:37.987905 3850 consensus_queue.cc:260] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } }
I20260808 10:07:37.987947 3849 consensus_queue.cc:260] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } }
I20260808 10:07:37.988180 3850 ts_tablet_manager.cc:1428] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 10:07:37.988265 3849 ts_tablet_manager.cc:1428] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:07:37.988253 3681 heartbeater.cc:503] Master 127.2.30.190:46551 was elected leader, sending a full tablet report...
I20260808 10:07:37.988351 3825 heartbeater.cc:503] Master 127.2.30.190:46551 was elected leader, sending a full tablet report...
I20260808 10:07:37.988471 3853 tablet_bootstrap.cc:492] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd: Bootstrap starting.
I20260808 10:07:37.989061 3853 tablet_bootstrap.cc:654] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:37.989463 3853 log.cc:824] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:37.990298 3853 tablet_bootstrap.cc:492] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd: No bootstrap required, opened a new log
I20260808 10:07:37.990353 3853 ts_tablet_manager.cc:1397] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:07:37.991830 3853 raft_consensus.cc:348] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } }
I20260808 10:07:37.991991 3853 raft_consensus.cc:374] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:37.992017 3853 raft_consensus.cc:729] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bcb3b5c2b6d41058729a11014eb16cd, State: Initialized, Role: FOLLOWER
I20260808 10:07:37.992105 3853 consensus_queue.cc:260] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } }
I20260808 10:07:37.992367 3853 ts_tablet_manager.cc:1428] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:07:37.992399 3536 heartbeater.cc:503] Master 127.2.30.190:46551 was elected leader, sending a full tablet report...
I20260808 10:07:37.994757 3857 raft_consensus.cc:482] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:07:37.994863 3857 raft_consensus.cc:504] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } }
I20260808 10:07:37.995215 3857 leader_election.cc:288] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c45f107da5df4ccc9a213e419d2f2c04 (127.2.30.131:45015), db01a79db2754211a161c93f5cee35e9 (127.2.30.130:45869)
I20260808 10:07:37.998360 3424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.995452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:45015 (local address 127.2.30.129:58635)
0808 10:07:37.995521 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:37.995531 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:37.995590 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:37.995756 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:37.995757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:37.995759 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:37.995818 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.995820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.996628 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.996629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.997443 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.997446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.998080 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.998081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.998169 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.998174 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:07:37.998235 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 10:07:37.998277 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 10:07:37.998420 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.995524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:58635 (local address 127.2.30.131:45015)
0808 10:07:37.995591 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:37.995593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:37.995604 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:07:37.995652 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:37.995653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:37.995666 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:37.995752 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:37.995883 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.996541 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.997545 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.997991 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.998049 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.998056 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:37.998306 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 10:07:37.998347 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:07:37.998643 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.995523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:50945 (local address 127.2.30.130:45869)
0808 10:07:37.995566 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:37.995568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:37.995923 (+ 355us) server_negotiation.cc:407] Connection header received
0808 10:07:37.995989 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:37.995990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:37.996001 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:37.996039 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:37.996182 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.996710 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.997717 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.998130 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.998335 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.998345 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:37.998517 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 10:07:37.998571 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:07:37.998744 3779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "159d958ceae444478dda5026fefa6d32" candidate_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c45f107da5df4ccc9a213e419d2f2c04" is_pre_election: true
I20260808 10:07:37.998926 3779 raft_consensus.cc:2454] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bcb3b5c2b6d41058729a11014eb16cd in term 0.
I20260808 10:07:37.998940 3858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:37.995452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:45869 (local address 127.2.30.129:50945)
0808 10:07:37.995840 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:37.995850 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:37.995933 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:37.996055 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:37.996056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:37.996057 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:37.996125 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.996127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.996790 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.996790 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.997611 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:37.997613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:37.998336 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:37.998337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:37.998408 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:37.998412 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:07:37.998755 (+ 343us) client_negotiation.cc:240] Negotiation successful
0808 10:07:37.998805 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":200,"thread_start_us":146,"threads_started":1}
I20260808 10:07:37.999249 3419 leader_election.cc:302] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bcb3b5c2b6d41058729a11014eb16cd, c45f107da5df4ccc9a213e419d2f2c04; no voters:
I20260808 10:07:37.999387 3857 raft_consensus.cc:2781] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:07:37.999396 3635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "159d958ceae444478dda5026fefa6d32" candidate_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db01a79db2754211a161c93f5cee35e9" is_pre_election: true
I20260808 10:07:37.999449 3857 raft_consensus.cc:482] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:07:37.999490 3857 raft_consensus.cc:3037] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:37.999543 3635 raft_consensus.cc:2454] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bcb3b5c2b6d41058729a11014eb16cd in term 0.
I20260808 10:07:38.000701 3857 raft_consensus.cc:504] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } }
I20260808 10:07:38.000905 3857 leader_election.cc:288] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [CANDIDATE]: Term 1 election: Requested vote from peers c45f107da5df4ccc9a213e419d2f2c04 (127.2.30.131:45015), db01a79db2754211a161c93f5cee35e9 (127.2.30.130:45869)
I20260808 10:07:38.000975 3635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "159d958ceae444478dda5026fefa6d32" candidate_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db01a79db2754211a161c93f5cee35e9"
I20260808 10:07:38.001055 3635 raft_consensus.cc:3037] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:38.001784 3635 raft_consensus.cc:2454] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bcb3b5c2b6d41058729a11014eb16cd in term 1.
I20260808 10:07:38.001885 3779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "159d958ceae444478dda5026fefa6d32" candidate_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c45f107da5df4ccc9a213e419d2f2c04"
I20260808 10:07:38.001996 3418 leader_election.cc:302] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bcb3b5c2b6d41058729a11014eb16cd, db01a79db2754211a161c93f5cee35e9; no voters:
I20260808 10:07:38.001986 3779 raft_consensus.cc:3037] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:38.002105 3857 raft_consensus.cc:2781] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:38.002290 3857 raft_consensus.cc:686] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [term 1 LEADER]: Becoming Leader. State: Replica: 3bcb3b5c2b6d41058729a11014eb16cd, State: Running, Role: LEADER
I20260808 10:07:38.002408 3857 consensus_queue.cc:237] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } }
I20260808 10:07:38.002964 3779 raft_consensus.cc:2454] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bcb3b5c2b6d41058729a11014eb16cd in term 1.
I20260808 10:07:38.003424 3326 catalog_manager.cc:5495] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bcb3b5c2b6d41058729a11014eb16cd (127.2.30.129). New cstate: current_term: 1 leader_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } health_report { overall_health: HEALTHY } } }
I20260808 10:07:38.011353 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:38.008895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49530 (local address 127.2.30.129:39907)
0808 10:07:38.008938 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:38.008940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:38.008953 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:07:38.009048 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:38.009049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:38.009068 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:38.009124 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:38.009238 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.009740 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.010526 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.010699 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.010747 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:38.010946 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:38.011104 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:38.011166 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:38.011220 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 10:07:38.011240 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:07:38.013219 3635 raft_consensus.cc:1261] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9 [term 1 FOLLOWER]: Refusing update from remote peer 3bcb3b5c2b6d41058729a11014eb16cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:38.013216 3779 raft_consensus.cc:1261] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04 [term 1 FOLLOWER]: Refusing update from remote peer 3bcb3b5c2b6d41058729a11014eb16cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:38.013589 3857 consensus_queue.cc:1035] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:38.013756 3859 consensus_queue.cc:1035] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:07:38.014726 3682 tablet.cc:2369] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:07:38.016877 3863 mvcc.cc:205] Tried to move back new op lower bound from 7316208263219429376 to 7316208263178625024. Current Snapshot: MvccSnapshot[applied={T|T < 7316208263219429376}]
I20260808 10:07:38.020094 3326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:07:38.020256 3326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:07:38.021219 3326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1: Sending DeleteTablet for 3 replicas of tablet 159d958ceae444478dda5026fefa6d32
I20260808 10:07:38.021585 3467 tablet_service.cc:1514] Processing DeleteTablet for tablet 159d958ceae444478dda5026fefa6d32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:38 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:49524
I20260808 10:07:38.021588 3758 tablet_service.cc:1514] Processing DeleteTablet for tablet 159d958ceae444478dda5026fefa6d32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:38 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:39276
I20260808 10:07:38.021865 3873 tablet_replica.cc:330] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04: stopping tablet replica
I20260808 10:07:38.021885 3872 tablet_replica.cc:330] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd: stopping tablet replica
I20260808 10:07:38.021997 3873 raft_consensus.cc:2229] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:07:38.022121 3873 raft_consensus.cc:2258] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:38.022114 3872 raft_consensus.cc:2229] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:07:38.022094 3614 tablet_service.cc:1514] Processing DeleteTablet for tablet 159d958ceae444478dda5026fefa6d32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:38 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:39974
I20260808 10:07:38.022297 3872 raft_consensus.cc:2258] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:38.022589 3326 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:07:38.022749 3326 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:07:38.022894 3872 ts_tablet_manager.cc:1905] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:38.023084 3874 tablet_replica.cc:330] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9: stopping tablet replica
I20260808 10:07:38.023612 3874 raft_consensus.cc:2229] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:07:38.023811 3874 raft_consensus.cc:2258] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:38.024106 3758 tablet_service.cc:1467] Processing CreateTablet for tablet fb2e918a36fc417abb413bbbc0aac159 (DEFAULT_TABLE table=test-table [id=4571d8289ca74857ae791defa4197007]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:38.024178 3467 tablet_service.cc:1467] Processing CreateTablet for tablet fb2e918a36fc417abb413bbbc0aac159 (DEFAULT_TABLE table=test-table [id=4571d8289ca74857ae791defa4197007]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:38.024225 3758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb2e918a36fc417abb413bbbc0aac159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:38.024303 3467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb2e918a36fc417abb413bbbc0aac159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:38.024935 3872 ts_tablet_manager.cc:1918] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:07:38.025004 3872 log.cc:1196] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/wal/wals/159d958ceae444478dda5026fefa6d32
I20260808 10:07:38.025290 3872 ts_tablet_manager.cc:1939] T 159d958ceae444478dda5026fefa6d32 P 3bcb3b5c2b6d41058729a11014eb16cd: Deleting consensus metadata
I20260808 10:07:38.025629 3853 tablet_bootstrap.cc:492] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd: Bootstrap starting.
I20260808 10:07:38.025743 3849 tablet_bootstrap.cc:492] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04: Bootstrap starting.
I20260808 10:07:38.025729 3309 catalog_manager.cc:4841] TS 3bcb3b5c2b6d41058729a11014eb16cd (127.2.30.129:39907): tablet 159d958ceae444478dda5026fefa6d32 (table test-table [id=8f5607e3d0f04c3699b06e6d1bdc4bc6]) successfully deleted
I20260808 10:07:38.026078 3853 tablet_bootstrap.cc:654] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:38.026340 3849 tablet_bootstrap.cc:654] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:38.026618 3874 ts_tablet_manager.cc:1905] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:38.026244 3873 ts_tablet_manager.cc:1905] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:38.026777 3614 tablet_service.cc:1467] Processing CreateTablet for tablet fb2e918a36fc417abb413bbbc0aac159 (DEFAULT_TABLE table=test-table [id=4571d8289ca74857ae791defa4197007]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:38.027063 3614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb2e918a36fc417abb413bbbc0aac159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:38.028250 3873 ts_tablet_manager.cc:1918] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:07:38.028333 3873 log.cc:1196] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/159d958ceae444478dda5026fefa6d32
I20260808 10:07:38.028570 3850 tablet_bootstrap.cc:492] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9: Bootstrap starting.
I20260808 10:07:38.028625 3873 ts_tablet_manager.cc:1939] T 159d958ceae444478dda5026fefa6d32 P c45f107da5df4ccc9a213e419d2f2c04: Deleting consensus metadata
I20260808 10:07:38.029021 3850 tablet_bootstrap.cc:654] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:38.029103 3312 catalog_manager.cc:4841] TS c45f107da5df4ccc9a213e419d2f2c04 (127.2.30.131:45015): tablet 159d958ceae444478dda5026fefa6d32 (table test-table [id=8f5607e3d0f04c3699b06e6d1bdc4bc6]) successfully deleted
I20260808 10:07:38.029412 3853 tablet_bootstrap.cc:492] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd: No bootstrap required, opened a new log
I20260808 10:07:38.029464 3853 ts_tablet_manager.cc:1397] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 10:07:38.029516 3849 tablet_bootstrap.cc:492] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04: No bootstrap required, opened a new log
I20260808 10:07:38.029564 3849 ts_tablet_manager.cc:1397] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 10:07:38.029628 3853 raft_consensus.cc:348] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.029712 3853 raft_consensus.cc:374] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:38.029742 3853 raft_consensus.cc:729] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bcb3b5c2b6d41058729a11014eb16cd, State: Initialized, Role: FOLLOWER
I20260808 10:07:38.029721 3849 raft_consensus.cc:348] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.029788 3853 consensus_queue.cc:260] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.029798 3849 raft_consensus.cc:374] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:38.029840 3849 raft_consensus.cc:729] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c45f107da5df4ccc9a213e419d2f2c04, State: Initialized, Role: FOLLOWER
I20260808 10:07:38.029884 3853 ts_tablet_manager.cc:1428] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:07:38.029892 3849 consensus_queue.cc:260] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.029989 3849 ts_tablet_manager.cc:1428] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:38.031337 3850 tablet_bootstrap.cc:492] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9: No bootstrap required, opened a new log
I20260808 10:07:38.031426 3850 ts_tablet_manager.cc:1397] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 10:07:38.031584 3850 raft_consensus.cc:348] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.031661 3850 raft_consensus.cc:374] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:38.031695 3850 raft_consensus.cc:729] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db01a79db2754211a161c93f5cee35e9, State: Initialized, Role: FOLLOWER
I20260808 10:07:38.031749 3850 consensus_queue.cc:260] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.031838 3850 ts_tablet_manager.cc:1428] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:07:38.032382 3874 ts_tablet_manager.cc:1918] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:07:38.032478 3874 log.cc:1196] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/159d958ceae444478dda5026fefa6d32
I20260808 10:07:38.032720 3874 ts_tablet_manager.cc:1939] T 159d958ceae444478dda5026fefa6d32 P db01a79db2754211a161c93f5cee35e9: Deleting consensus metadata
I20260808 10:07:38.033159 3311 catalog_manager.cc:4841] TS db01a79db2754211a161c93f5cee35e9 (127.2.30.130:45869): tablet 159d958ceae444478dda5026fefa6d32 (table test-table [id=8f5607e3d0f04c3699b06e6d1bdc4bc6]) successfully deleted
I20260808 10:07:38.071040 3857 raft_consensus.cc:482] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:07:38.071185 3857 raft_consensus.cc:504] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.071375 3857 leader_election.cc:288] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db01a79db2754211a161c93f5cee35e9 (127.2.30.130:45869), c45f107da5df4ccc9a213e419d2f2c04 (127.2.30.131:45015)
I20260808 10:07:38.071630 3635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb2e918a36fc417abb413bbbc0aac159" candidate_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db01a79db2754211a161c93f5cee35e9" is_pre_election: true
I20260808 10:07:38.071756 3635 raft_consensus.cc:2454] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bcb3b5c2b6d41058729a11014eb16cd in term 0.
I20260808 10:07:38.071689 3779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb2e918a36fc417abb413bbbc0aac159" candidate_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c45f107da5df4ccc9a213e419d2f2c04" is_pre_election: true
I20260808 10:07:38.071810 3779 raft_consensus.cc:2454] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bcb3b5c2b6d41058729a11014eb16cd in term 0.
I20260808 10:07:38.071991 3418 leader_election.cc:302] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bcb3b5c2b6d41058729a11014eb16cd, db01a79db2754211a161c93f5cee35e9; no voters:
I20260808 10:07:38.072098 3857 raft_consensus.cc:2781] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:07:38.072139 3857 raft_consensus.cc:482] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:07:38.072170 3857 raft_consensus.cc:3037] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:38.072932 3857 raft_consensus.cc:504] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.073096 3857 leader_election.cc:288] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [CANDIDATE]: Term 1 election: Requested vote from peers db01a79db2754211a161c93f5cee35e9 (127.2.30.130:45869), c45f107da5df4ccc9a213e419d2f2c04 (127.2.30.131:45015)
I20260808 10:07:38.073252 3635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb2e918a36fc417abb413bbbc0aac159" candidate_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db01a79db2754211a161c93f5cee35e9"
I20260808 10:07:38.073282 3779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb2e918a36fc417abb413bbbc0aac159" candidate_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c45f107da5df4ccc9a213e419d2f2c04"
I20260808 10:07:38.073340 3635 raft_consensus.cc:3037] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:38.073372 3779 raft_consensus.cc:3037] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:38.074172 3779 raft_consensus.cc:2454] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bcb3b5c2b6d41058729a11014eb16cd in term 1.
I20260808 10:07:38.074172 3635 raft_consensus.cc:2454] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bcb3b5c2b6d41058729a11014eb16cd in term 1.
I20260808 10:07:38.074361 3418 leader_election.cc:302] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bcb3b5c2b6d41058729a11014eb16cd, db01a79db2754211a161c93f5cee35e9; no voters:
I20260808 10:07:38.074491 3857 raft_consensus.cc:2781] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:38.074566 3857 raft_consensus.cc:686] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [term 1 LEADER]: Becoming Leader. State: Replica: 3bcb3b5c2b6d41058729a11014eb16cd, State: Running, Role: LEADER
I20260808 10:07:38.074615 3857 consensus_queue.cc:237] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.075255 3326 catalog_manager.cc:5495] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bcb3b5c2b6d41058729a11014eb16cd (127.2.30.129). New cstate: current_term: 1 leader_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } health_report { overall_health: UNKNOWN } } }
W20260808 10:07:38.082710 3537 tablet.cc:2369] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 10:07:38 dist-test-slave-w206 krb5kdc[3279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183657, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:38.120033 3885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:38.111813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46551 (local address 127.0.0.1:60746)
0808 10:07:38.111970 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:38.111983 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:38.112055 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:38.112211 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:38.112212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:38.112507 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:38.112564 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:38.112566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.113510 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.113511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:38.114057 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:38.114059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.114492 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.114493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:38.114534 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:38.115210 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:38.115215 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:38.116990 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:38.118883 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:38.118891 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:38.118893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:38.119174 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:38.119388 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:38.119389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:38.119389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:38.119452 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:38.119725 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:38.119731 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:38.119775 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 10:07:38.119827 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 10:07:38.119954 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260808 10:07:38.120148 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:38.111892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60746 (local address 127.2.30.190:46551)
0808 10:07:38.111957 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:38.111960 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:38.112058 (+ 98us) server_negotiation.cc:407] Connection header received
0808 10:07:38.112098 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:38.112099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:38.112112 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:38.112199 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:38.112646 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.113408 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.114144 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.114409 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.114485 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:38.117086 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:38.117087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:38.117088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:38.117095 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:38.118786 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:38.119248 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:38.119250 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:38.119250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:38.119316 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:38.119516 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:38.119517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:38.119517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:38.119661 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:38.119732 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:38.119886 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 10:07:38.120030 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":33}
I20260808 10:07:38.122541 3326 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:60746:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 10:07:38.122951 3326 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:07:38.124667 3758 tablet_service.cc:1467] Processing CreateTablet for tablet d811f8cc53684ffa918a68498e84f0b2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ec53b16610864b4b861e674e4b84d38a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:07:38.124648 3614 tablet_service.cc:1467] Processing CreateTablet for tablet d811f8cc53684ffa918a68498e84f0b2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ec53b16610864b4b861e674e4b84d38a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:07:38.124840 3758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d811f8cc53684ffa918a68498e84f0b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:38.124868 3614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d811f8cc53684ffa918a68498e84f0b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:38.126145 3850 tablet_bootstrap.cc:492] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9: Bootstrap starting.
I20260808 10:07:38.126477 3849 tablet_bootstrap.cc:492] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04: Bootstrap starting.
I20260808 10:07:38.126643 3850 tablet_bootstrap.cc:654] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:38.126791 3467 tablet_service.cc:1467] Processing CreateTablet for tablet d811f8cc53684ffa918a68498e84f0b2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ec53b16610864b4b861e674e4b84d38a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:07:38.126936 3849 tablet_bootstrap.cc:654] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:38.126948 3467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d811f8cc53684ffa918a68498e84f0b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:38.127687 3849 tablet_bootstrap.cc:492] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04: No bootstrap required, opened a new log
I20260808 10:07:38.127763 3849 ts_tablet_manager.cc:1397] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:38.127946 3849 raft_consensus.cc:348] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.128028 3849 raft_consensus.cc:374] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:38.128057 3849 raft_consensus.cc:729] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c45f107da5df4ccc9a213e419d2f2c04, State: Initialized, Role: FOLLOWER
I20260808 10:07:38.128103 3849 consensus_queue.cc:260] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.128201 3849 ts_tablet_manager.cc:1428] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:07:38.128337 3855 tablet_replica.cc:439] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } }
I20260808 10:07:38.128423 3855 tablet_replica.cc:442] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:07:38.128634 3853 tablet_bootstrap.cc:492] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd: Bootstrap starting.
I20260808 10:07:38.129071 3853 tablet_bootstrap.cc:654] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:38.129751 3853 tablet_bootstrap.cc:492] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd: No bootstrap required, opened a new log
I20260808 10:07:38.129809 3853 ts_tablet_manager.cc:1397] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:38.129964 3853 raft_consensus.cc:348] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.130048 3853 raft_consensus.cc:374] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:38.130069 3853 raft_consensus.cc:729] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bcb3b5c2b6d41058729a11014eb16cd, State: Initialized, Role: FOLLOWER
I20260808 10:07:38.130111 3853 consensus_queue.cc:260] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.130205 3853 ts_tablet_manager.cc:1428] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:07:38.130317 3857 tablet_replica.cc:439] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } }
I20260808 10:07:38.130388 3857 tablet_replica.cc:442] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:07:38.130888 3850 tablet_bootstrap.cc:492] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9: No bootstrap required, opened a new log
I20260808 10:07:38.130962 3850 ts_tablet_manager.cc:1397] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260808 10:07:38.131294 3850 raft_consensus.cc:348] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.131348 3850 raft_consensus.cc:374] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:38.131361 3850 raft_consensus.cc:729] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db01a79db2754211a161c93f5cee35e9, State: Initialized, Role: FOLLOWER
I20260808 10:07:38.131423 3850 consensus_queue.cc:260] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.131531 3850 ts_tablet_manager.cc:1428] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:38.131613 3854 tablet_replica.cc:439] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } }
I20260808 10:07:38.131714 3854 tablet_replica.cc:442] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 10:07:38.184152 3826 tablet.cc:2369] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:07:38.238034 3854 raft_consensus.cc:482] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:07:38.238161 3854 raft_consensus.cc:504] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.238606 3854 leader_election.cc:288] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bcb3b5c2b6d41058729a11014eb16cd (127.2.30.129:39907), c45f107da5df4ccc9a213e419d2f2c04 (127.2.30.131:45015)
I20260808 10:07:38.242172 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:38.239218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:42133 (local address 127.2.30.131:45015)
0808 10:07:38.239275 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:38.239278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:38.239287 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:07:38.239320 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:38.239321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:38.239331 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:38.239389 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:38.239595 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.240416 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.241408 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.241762 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.241858 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:38.241878 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:38.242053 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 10:07:38.242099 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 10:07:38.242174 3570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:38.238891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:45015 (local address 127.2.30.130:42133)
0808 10:07:38.239100 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:38.239110 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:38.239173 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:38.239441 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:38.239442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:38.239444 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:38.239515 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:38.239517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.240517 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.240518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:38.241313 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:38.241316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.241856 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.241857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:38.241956 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:38.241965 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:07:38.242043 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 10:07:38.242089 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":121,"threads_started":1}
I20260808 10:07:38.242511 3779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d811f8cc53684ffa918a68498e84f0b2" candidate_uuid: "db01a79db2754211a161c93f5cee35e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c45f107da5df4ccc9a213e419d2f2c04" is_pre_election: true
I20260808 10:07:38.242622 3779 raft_consensus.cc:2454] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db01a79db2754211a161c93f5cee35e9 in term 0.
I20260808 10:07:38.242791 3565 leader_election.cc:302] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c45f107da5df4ccc9a213e419d2f2c04, db01a79db2754211a161c93f5cee35e9; no voters:
I20260808 10:07:38.242962 3854 raft_consensus.cc:2781] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:07:38.243040 3854 raft_consensus.cc:482] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:07:38.243075 3854 raft_consensus.cc:3037] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:38.243854 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:38.239274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:46211 (local address 127.2.30.129:39907)
0808 10:07:38.239344 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:38.239346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:38.239358 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:07:38.239392 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:38.239394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:38.239405 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:38.239472 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:38.239712 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.240431 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.243021 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.243469 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.243563 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:38.243572 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:38.243735 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 10:07:38.243781 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 10:07:38.243896 3889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:38.238796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:39907 (local address 127.2.30.130:46211)
0808 10:07:38.239089 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:38.239120 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:38.239173 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:38.239526 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:38.239527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:38.239529 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:38.239627 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:38.239629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.241405 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.241405 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:38.242873 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:38.242876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.243563 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.243564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:38.243655 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:38.243659 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:07:38.243728 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 10:07:38.243770 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272}
I20260808 10:07:38.243829 3854 raft_consensus.cc:504] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.244012 3854 leader_election.cc:288] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [CANDIDATE]: Term 1 election: Requested vote from peers 3bcb3b5c2b6d41058729a11014eb16cd (127.2.30.129:39907), c45f107da5df4ccc9a213e419d2f2c04 (127.2.30.131:45015)
I20260808 10:07:38.244179 3779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d811f8cc53684ffa918a68498e84f0b2" candidate_uuid: "db01a79db2754211a161c93f5cee35e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c45f107da5df4ccc9a213e419d2f2c04"
I20260808 10:07:38.244284 3779 raft_consensus.cc:3037] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:38.244305 3487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d811f8cc53684ffa918a68498e84f0b2" candidate_uuid: "db01a79db2754211a161c93f5cee35e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" is_pre_election: true
I20260808 10:07:38.244308 3486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d811f8cc53684ffa918a68498e84f0b2" candidate_uuid: "db01a79db2754211a161c93f5cee35e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bcb3b5c2b6d41058729a11014eb16cd"
I20260808 10:07:38.244463 3487 raft_consensus.cc:2454] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db01a79db2754211a161c93f5cee35e9 in term 0.
I20260808 10:07:38.244975 3779 raft_consensus.cc:2454] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db01a79db2754211a161c93f5cee35e9 in term 1.
I20260808 10:07:38.245141 3565 leader_election.cc:302] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c45f107da5df4ccc9a213e419d2f2c04, db01a79db2754211a161c93f5cee35e9; no voters: 3bcb3b5c2b6d41058729a11014eb16cd
I20260808 10:07:38.245254 3854 raft_consensus.cc:2781] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:38.245428 3854 raft_consensus.cc:686] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [term 1 LEADER]: Becoming Leader. State: Replica: db01a79db2754211a161c93f5cee35e9, State: Running, Role: LEADER
I20260808 10:07:38.245517 3854 consensus_queue.cc:237] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } }
I20260808 10:07:38.245915 3890 tablet_replica.cc:439] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9: TxnStatusTablet state changed. Reason: New leader db01a79db2754211a161c93f5cee35e9. Latest consensus state: current_term: 1 leader_uuid: "db01a79db2754211a161c93f5cee35e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } }
I20260808 10:07:38.246016 3890 tablet_replica.cc:442] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:07:38.246222 3892 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 10:07:38.246361 3892 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 10:07:38.246333 3326 catalog_manager.cc:5495] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to db01a79db2754211a161c93f5cee35e9 (127.2.30.130). New cstate: current_term: 1 leader_uuid: "db01a79db2754211a161c93f5cee35e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } health_report { overall_health: UNKNOWN } } }
I20260808 10:07:38.253399 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:38.250396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39984 (local address 127.2.30.130:45869)
0808 10:07:38.250458 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:38.250460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:38.250470 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:07:38.250531 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:38.250532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:38.250550 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:38.250628 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:38.250772 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.251382 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.252581 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.252776 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.252843 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:38.253064 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:38.253162 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:38.253221 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:38.253318 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 10:07:38.253340 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:07:38.253448 3885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:38.250252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:45869 (local address 127.0.0.1:39984)
0808 10:07:38.250389 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:38.250402 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:38.250450 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:38.250643 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:38.250644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:38.250646 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:07:38.250703 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:38.250706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.251467 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.251469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:38.252416 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:38.252420 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.252860 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.252861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:38.252933 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:38.252956 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:07:38.253233 (+ 277us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:07:38.253244 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 10:07:38.253300 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 10:07:38.253359 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260808 10:07:38.254828 3779 raft_consensus.cc:1261] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04 [term 1 FOLLOWER]: Refusing update from remote peer db01a79db2754211a161c93f5cee35e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:38.254894 3486 raft_consensus.cc:3037] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:38.255131 3854 consensus_queue.cc:1035] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:38.255847 3855 tablet_replica.cc:439] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04: TxnStatusTablet state changed. Reason: New leader db01a79db2754211a161c93f5cee35e9. Latest consensus state: current_term: 1 leader_uuid: "db01a79db2754211a161c93f5cee35e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } }
I20260808 10:07:38.255950 3855 tablet_replica.cc:442] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:07:38.255993 3486 raft_consensus.cc:1261] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd [term 1 FOLLOWER]: Refusing update from remote peer db01a79db2754211a161c93f5cee35e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:38.256348 3854 consensus_queue.cc:1035] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:38.257092 3857 tablet_replica.cc:439] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd: TxnStatusTablet state changed. Reason: New leader db01a79db2754211a161c93f5cee35e9. Latest consensus state: current_term: 1 leader_uuid: "db01a79db2754211a161c93f5cee35e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } }
I20260808 10:07:38.257190 3857 tablet_replica.cc:442] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:07:38.257253 3854 tablet_replica.cc:439] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "db01a79db2754211a161c93f5cee35e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } }
I20260808 10:07:38.257330 3854 tablet_replica.cc:442] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:07:38.257835 3890 tablet_replica.cc:439] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "db01a79db2754211a161c93f5cee35e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } }
I20260808 10:07:38.257921 3890 tablet_replica.cc:442] T d811f8cc53684ffa918a68498e84f0b2 P db01a79db2754211a161c93f5cee35e9: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:07:38.258239 3862 mvcc.cc:205] Tried to move back new op lower bound from 7316208264209645568 to 7316208264174391296. Current Snapshot: MvccSnapshot[applied={T|T < 7316208264209645568}]
I20260808 10:07:38.258385 3864 mvcc.cc:205] Tried to move back new op lower bound from 7316208264209645568 to 7316208264174391296. Current Snapshot: MvccSnapshot[applied={T|T < 7316208264209645568}]
I20260808 10:07:38.258636 3855 tablet_replica.cc:439] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "db01a79db2754211a161c93f5cee35e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } }
I20260808 10:07:38.258714 3855 tablet_replica.cc:442] T d811f8cc53684ffa918a68498e84f0b2 P c45f107da5df4ccc9a213e419d2f2c04: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:07:38.258776 3857 tablet_replica.cc:439] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "db01a79db2754211a161c93f5cee35e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bcb3b5c2b6d41058729a11014eb16cd" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39907 } } peers { permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 } } peers { permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 } } }
I20260808 10:07:38.258869 3857 tablet_replica.cc:442] T d811f8cc53684ffa918a68498e84f0b2 P 3bcb3b5c2b6d41058729a11014eb16cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:07:38.265355 3901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:38.262426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:45869 (local address 127.2.30.129:41573)
0808 10:07:38.262601 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:38.262614 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:38.262682 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:38.262807 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:38.262808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:38.262810 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:07:38.262866 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:38.262868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.263626 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.263628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:38.264330 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:38.264333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.264709 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.264710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:38.264779 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:38.264802 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:07:38.265160 (+ 358us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:07:38.265162 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:07:38.265225 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 10:07:38.265280 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":106,"threads_started":1}
I20260808 10:07:38.265345 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:38.262521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:41573 (local address 127.2.30.130:45869)
0808 10:07:38.262591 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:38.262593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:38.262665 (+ 72us) server_negotiation.cc:407] Connection header received
0808 10:07:38.262745 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:38.262746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:38.262758 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:38.262804 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:38.262920 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.263534 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.264422 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.264619 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.264707 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:38.264904 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:38.264993 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:38.265039 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:38.265232 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 10:07:38.265263 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:07:38.267786 3635 raft_consensus.cc:1261] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9 [term 1 FOLLOWER]: Refusing update from remote peer 3bcb3b5c2b6d41058729a11014eb16cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:38.267800 3779 raft_consensus.cc:1261] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04 [term 1 FOLLOWER]: Refusing update from remote peer 3bcb3b5c2b6d41058729a11014eb16cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:38.268100 3857 consensus_queue.cc:1035] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c45f107da5df4ccc9a213e419d2f2c04" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:38.268232 3859 consensus_queue.cc:1035] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "db01a79db2754211a161c93f5cee35e9" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 45869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:38.277846 3688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:38.274444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:39907 (local address 127.2.30.130:57707)
0808 10:07:38.274524 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:38.274537 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:38.274578 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:38.274778 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:38.274778 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:38.274779 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:07:38.274839 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:38.274841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.275874 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.275876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:38.276891 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:38.276895 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.277356 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.277357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:38.277428 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:38.277449 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:07:38.277645 (+ 196us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:07:38.277648 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:07:38.277699 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 10:07:38.277758 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 10:07:38.277849 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:38.274516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:57707 (local address 127.2.30.129:39907)
0808 10:07:38.274557 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:38.274559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:38.274568 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:07:38.274695 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:38.274696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:38.274709 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:38.274763 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:38.274889 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.275668 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.276985 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.277250 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.277314 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:38.277551 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:38.277590 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:38.277642 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:38.277761 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 10:07:38.277786 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:07:38.287774 3325 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:07:38.287912 3325 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:07:38.288682 3325 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6baa28ca53a6419ea32a731e0b4d79b1: Sending DeleteTablet for 3 replicas of tablet fb2e918a36fc417abb413bbbc0aac159
I20260808 10:07:38.288972 3467 tablet_service.cc:1514] Processing DeleteTablet for tablet fb2e918a36fc417abb413bbbc0aac159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:38 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:49524
I20260808 10:07:38.289027 3758 tablet_service.cc:1514] Processing DeleteTablet for tablet fb2e918a36fc417abb413bbbc0aac159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:38 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:39276
I20260808 10:07:38.289135 3872 tablet_replica.cc:330] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd: stopping tablet replica
I20260808 10:07:38.289116 3614 tablet_service.cc:1514] Processing DeleteTablet for tablet fb2e918a36fc417abb413bbbc0aac159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:38 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:39974
I20260808 10:07:38.289216 3874 tablet_replica.cc:330] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9: stopping tablet replica
I20260808 10:07:38.289211 3872 raft_consensus.cc:2229] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:07:38.289265 3874 raft_consensus.cc:2229] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:07:38.289299 3872 raft_consensus.cc:2258] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:38.289309 3874 raft_consensus.cc:2258] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:38.289393 3873 tablet_replica.cc:330] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04: stopping tablet replica
I20260808 10:07:38.289456 3873 raft_consensus.cc:2229] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:07:38.289501 3873 raft_consensus.cc:2258] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:38.289834 3872 ts_tablet_manager.cc:1905] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:38.289969 3873 ts_tablet_manager.cc:1905] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:38.291249 3874 ts_tablet_manager.cc:1905] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:38.291638 3873 ts_tablet_manager.cc:1918] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:07:38.291679 3872 ts_tablet_manager.cc:1918] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:07:38.291703 3873 log.cc:1196] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/fb2e918a36fc417abb413bbbc0aac159
I20260808 10:07:38.291742 3872 log.cc:1196] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-0/wal/wals/fb2e918a36fc417abb413bbbc0aac159
I20260808 10:07:38.291963 3873 ts_tablet_manager.cc:1939] T fb2e918a36fc417abb413bbbc0aac159 P c45f107da5df4ccc9a213e419d2f2c04: Deleting consensus metadata
I20260808 10:07:38.291981 3872 ts_tablet_manager.cc:1939] T fb2e918a36fc417abb413bbbc0aac159 P 3bcb3b5c2b6d41058729a11014eb16cd: Deleting consensus metadata
I20260808 10:07:38.292339 3309 catalog_manager.cc:4841] TS 3bcb3b5c2b6d41058729a11014eb16cd (127.2.30.129:39907): tablet fb2e918a36fc417abb413bbbc0aac159 (table test-table [id=4571d8289ca74857ae791defa4197007]) successfully deleted
I20260808 10:07:38.292352 3312 catalog_manager.cc:4841] TS c45f107da5df4ccc9a213e419d2f2c04 (127.2.30.131:45015): tablet fb2e918a36fc417abb413bbbc0aac159 (table test-table [id=4571d8289ca74857ae791defa4197007]) successfully deleted
I20260808 10:07:38.293069 3874 ts_tablet_manager.cc:1918] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:07:38.293145 3874 log.cc:1196] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/fb2e918a36fc417abb413bbbc0aac159
I20260808 10:07:38.293394 3874 ts_tablet_manager.cc:1939] T fb2e918a36fc417abb413bbbc0aac159 P db01a79db2754211a161c93f5cee35e9: Deleting consensus metadata
I20260808 10:07:38.293926 3311 catalog_manager.cc:4841] TS db01a79db2754211a161c93f5cee35e9 (127.2.30.130:45869): tablet fb2e918a36fc417abb413bbbc0aac159 (table test-table [id=4571d8289ca74857ae791defa4197007]) successfully deleted
Aug 08 10:07:38 dist-test-slave-w206 krb5kdc[3279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183657, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.30.129@KRBTEST.COM
I20260808 10:07:38.302964 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:38.291413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49538 (local address 127.2.30.129:39907)
0808 10:07:38.291456 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:38.291458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:38.292513 (+ 1055us) server_negotiation.cc:407] Connection header received
0808 10:07:38.292615 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:38.292616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:38.292630 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:38.292660 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:38.293534 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.294333 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.295512 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.295704 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.295776 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:38.299658 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:38.299671 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:38.299671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:38.299682 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:38.301486 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:38.301954 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:38.301958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:38.301958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:38.302032 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:38.302305 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:38.302306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:38.302306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:38.302488 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:38.302612 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:38.302741 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 10:07:38.302831 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 10:07:38.303306 3530 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49538
I20260808 10:07:38.311664 3377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:38.304494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60756 (local address 127.2.30.190:46551)
0808 10:07:38.304564 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:38.304567 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:38.304681 (+ 114us) server_negotiation.cc:407] Connection header received
0808 10:07:38.304778 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:38.304779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:38.304791 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:38.304846 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:38.305493 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.306053 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.306832 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:38.307150 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:38.307255 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:38.308598 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:38.308600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:38.308600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:38.308609 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:38.310326 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:38.310731 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:38.310733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:38.310733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:38.310803 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:38.311107 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:38.311108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:38.311109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:38.311238 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:38.311359 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:38.311468 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 10:07:38.311555 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 10:07:38.311944 3325 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60756
I20260808 10:07:38.312649 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 3399
I20260808 10:07:38.323196 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 3545
I20260808 10:07:38.330296 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 3690
I20260808 10:07:38.336342 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 3292
2026-08-08T10:07:38Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2334 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:38 dist-test-slave-w206 krb5kdc[3913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:38 dist-test-slave-w206 krb5kdc[3913](info): set up 2 sockets
Aug 08 10:07:38 dist-test-slave-w206 krb5kdc[3913](info): commencing operation
krb5kdc: starting...
W20260808 10:07:39.418473 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:07:39 dist-test-slave-w206 krb5kdc[3913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:07:39.555262 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:42393
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:42393
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:39.639590 3926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:39.639827 3926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:39.639853 3926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:39.641054 3926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:39.641096 3926 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 10:07:39.641110 3926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:39.641124 3926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:39.641135 3926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:39.641147 3926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:39.642658 3926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:42393
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:42393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:39.642876 3926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:39.643101 3926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:39.648272 3934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:39.648283 3932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:39.648265 3931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:39.648564 3926 server_base.cc:1034] running on GCE node
I20260808 10:07:39.648825 3926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:39.649184 3926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:39.650348 3926 hybrid_clock.cc:648] HybridClock initialized: now 1786183659650329 us; error 44 us; skew 500 ppm
Aug 08 10:07:39 dist-test-slave-w206 krb5kdc[3913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183659, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:39.653111 3926 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:07:39.653643 3926 webserver.cc:460] Webserver started at http://127.2.30.190:43683/ using document root <none> and password file <none>
I20260808 10:07:39.653911 3926 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:39.653985 3926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:39.654143 3926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:39.655382 3926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "9bf3086c1ef64675bfdc48fe63433d6b"
format_stamp: "Formatted at 2026-08-08 10:07:39 on dist-test-slave-w206"
server_key: "529d21993b87861563d13ecca134d8e6"
server_key_iv: "e92dc227f9c526466f9a7b825b328e1a"
server_key_version: "encryptionkey@0"
I20260808 10:07:39.655776 3926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "9bf3086c1ef64675bfdc48fe63433d6b"
format_stamp: "Formatted at 2026-08-08 10:07:39 on dist-test-slave-w206"
server_key: "529d21993b87861563d13ecca134d8e6"
server_key_iv: "e92dc227f9c526466f9a7b825b328e1a"
server_key_version: "encryptionkey@0"
I20260808 10:07:39.657233 3926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:07:39.658067 3941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:39.658286 3926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:39.658402 3926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "9bf3086c1ef64675bfdc48fe63433d6b"
format_stamp: "Formatted at 2026-08-08 10:07:39 on dist-test-slave-w206"
server_key: "529d21993b87861563d13ecca134d8e6"
server_key_iv: "e92dc227f9c526466f9a7b825b328e1a"
server_key_version: "encryptionkey@0"
I20260808 10:07:39.658483 3926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:39.667375 3926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:39.669610 3926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:39.669791 3926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:39.674907 3926 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:42393
I20260808 10:07:39.674945 4004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:42393 every 8 connection(s)
I20260808 10:07:39.675335 3926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:39.676003 4005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:39.678366 4005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b: Bootstrap starting.
I20260808 10:07:39.679066 4005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:39.679375 4005 log.cc:824] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:39.680115 4005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b: No bootstrap required, opened a new log
I20260808 10:07:39.681511 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 3926
I20260808 10:07:39.681622 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:07:39.681789 2170 external_mini_cluster.cc:1428] Setting key 78b70bb311adac3f49fb14e68b1ef2cc
I20260808 10:07:39.682843 4005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf3086c1ef64675bfdc48fe63433d6b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 42393 } }
I20260808 10:07:39.683045 4005 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:39.683115 4005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bf3086c1ef64675bfdc48fe63433d6b, State: Initialized, Role: FOLLOWER
I20260808 10:07:39.683255 4005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf3086c1ef64675bfdc48fe63433d6b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 42393 } }
I20260808 10:07:39.683353 4005 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:39.683401 4005 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:39.683458 4005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:39.684082 4005 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf3086c1ef64675bfdc48fe63433d6b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 42393 } }
I20260808 10:07:39.684206 4005 leader_election.cc:302] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bf3086c1ef64675bfdc48fe63433d6b; no voters:
I20260808 10:07:39.684391 4005 leader_election.cc:288] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:39.684458 4010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:39.684629 4010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [term 1 LEADER]: Becoming Leader. State: Replica: 9bf3086c1ef64675bfdc48fe63433d6b, State: Running, Role: LEADER
I20260808 10:07:39.684705 4005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:39.684721 4010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf3086c1ef64675bfdc48fe63433d6b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 42393 } }
I20260808 10:07:39.685129 4010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bf3086c1ef64675bfdc48fe63433d6b. Latest consensus state: current_term: 1 leader_uuid: "9bf3086c1ef64675bfdc48fe63433d6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf3086c1ef64675bfdc48fe63433d6b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 42393 } } }
I20260808 10:07:39.685200 4010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:39.685343 4012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bf3086c1ef64675bfdc48fe63433d6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf3086c1ef64675bfdc48fe63433d6b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 42393 } } }
I20260808 10:07:39.685390 4012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:39.685722 4016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:39.686225 4016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:39.688107 4016 catalog_manager.cc:1347] Generated new cluster ID: 37989fd095cf4bd69df145fcd3852417
I20260808 10:07:39.688171 4016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:07:39 dist-test-slave-w206 krb5kdc[3913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:39.694258 4009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:39.682465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32810 (local address 127.2.30.190:42393)
0808 10:07:39.682705 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:39.682714 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:39.682742 (+ 28us) server_negotiation.cc:407] Connection header received
0808 10:07:39.682846 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:39.682852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:39.682898 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:39.683079 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:39.684048 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:39.685495 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:39.686589 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:39.687214 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:39.687330 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:39.690364 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:39.690369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:39.690369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:39.690382 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:39.692486 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:39.692929 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:39.692932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:39.692933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:39.693062 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:39.693307 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:39.693308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:39.693308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:39.693619 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:39.693774 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:39.693916 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 10:07:39.694053 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":97,"threads_started":1}
I20260808 10:07:39.706593 4016 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:39.707248 4016 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:39.714867 4016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b: Generated new TSK 0
I20260808 10:07:39.715133 4016 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:07:39.746750 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:42393
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:39.833541 4032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:39.833766 4032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:39.833794 4032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:39.835038 4032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:39.835086 4032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:39.835109 4032 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 10:07:39.835121 4032 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:07:39.835193 4032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:07:39.836826 4032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.30.190:42393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:39.837118 4032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:39.837375 4032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:39.842885 4038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:39.842861 4037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:39.842834 4040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:39.843133 4032 server_base.cc:1034] running on GCE node
I20260808 10:07:39.843331 4032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:39.843582 4032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:39.844731 4032 hybrid_clock.cc:648] HybridClock initialized: now 1786183659844710 us; error 38 us; skew 500 ppm
Aug 08 10:07:39 dist-test-slave-w206 krb5kdc[3913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183659, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:39.847625 4032 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:07:39.848205 4032 webserver.cc:460] Webserver started at http://127.2.30.129:42835/ using document root <none> and password file <none>
I20260808 10:07:39.848495 4032 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:39.848570 4032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:39.848731 4032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:39.849870 4032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "b693b00a8aee4f58b0248ad4b8a66021"
format_stamp: "Formatted at 2026-08-08 10:07:39 on dist-test-slave-w206"
server_key: "e4cbb1aaede8347672e6a3c087082d9d"
server_key_iv: "c93b5c97d54c477a8b0f012e54c7956c"
server_key_version: "encryptionkey@0"
I20260808 10:07:39.850296 4032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "b693b00a8aee4f58b0248ad4b8a66021"
format_stamp: "Formatted at 2026-08-08 10:07:39 on dist-test-slave-w206"
server_key: "e4cbb1aaede8347672e6a3c087082d9d"
server_key_iv: "c93b5c97d54c477a8b0f012e54c7956c"
server_key_version: "encryptionkey@0"
I20260808 10:07:39.852355 4032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 10:07:39.853178 4047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:39.853433 4032 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:39.853559 4032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "b693b00a8aee4f58b0248ad4b8a66021"
format_stamp: "Formatted at 2026-08-08 10:07:39 on dist-test-slave-w206"
server_key: "e4cbb1aaede8347672e6a3c087082d9d"
server_key_iv: "c93b5c97d54c477a8b0f012e54c7956c"
server_key_version: "encryptionkey@0"
I20260808 10:07:39.853639 4032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:39.863401 4032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:39.865599 4032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:39.865784 4032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:39.866391 4032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:39.866452 4032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:39.866489 4032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:39.866529 4032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:07:39 dist-test-slave-w206 krb5kdc[3913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183659, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:39.874259 4032 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:32903
I20260808 10:07:39.874636 4032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:07:39.874341 4165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:32903 every 8 connection(s)
I20260808 10:07:39.876734 4009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:39.866886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:46683 (local address 127.2.30.190:42393)
0808 10:07:39.866961 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:39.866963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:39.867193 (+ 230us) server_negotiation.cc:407] Connection header received
0808 10:07:39.867689 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:39.867691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:39.867712 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:39.867804 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:39.868412 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:39.869267 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:39.870489 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:39.870747 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:39.870834 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:39.873813 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:39.873815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:39.873815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:39.873822 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:39.875493 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:39.875871 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:39.875872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:39.875872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:39.875930 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:39.876148 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:39.876149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:39.876149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:39.876274 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:39.876330 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:39.876585 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 10:07:39.876660 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:07:39.876912 4056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:39.866867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:42393 (local address 127.2.30.129:46683)
0808 10:07:39.867045 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:39.867089 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:39.867589 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:39.867866 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:39.867869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:39.868106 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:39.868305 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:39.868309 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:39.869387 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:39.869390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:39.870325 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:39.870328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:39.870845 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:39.870847 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:39.871029 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:39.871700 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:39.871713 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:39.873710 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:39.875570 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:39.875575 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:39.875577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:39.875798 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:39.875992 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:39.875992 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:39.875992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:39.876091 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:39.876344 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:39.876350 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:39.876468 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 10:07:39.876554 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:07:39.876676 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":108,"threads_started":1}
I20260808 10:07:39.877604 4166 heartbeater.cc:347] Connected to a master server at 127.2.30.190:42393
I20260808 10:07:39.877708 4166 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:39.877926 4166 heartbeater.cc:511] Master 127.2.30.190:42393 requested a full tablet report, sending...
I20260808 10:07:39.878624 3957 ts_manager.cc:194] Registered new tserver with Master: b693b00a8aee4f58b0248ad4b8a66021 (127.2.30.129:32903)
I20260808 10:07:39.879494 3957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:46683
I20260808 10:07:39.883178 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 4032
I20260808 10:07:39.883302 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:07:39.883502 2170 external_mini_cluster.cc:1428] Setting key cee19b80c7c21e5c58cc89eaad2207b7
I20260808 10:07:39.884943 4009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:39.878792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:35635 (local address 127.2.30.190:42393)
0808 10:07:39.879068 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:39.879072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:39.879217 (+ 145us) server_negotiation.cc:407] Connection header received
0808 10:07:39.879275 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:39.879276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:39.879287 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:39.879355 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:39.879739 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:39.880293 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:39.881049 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:39.881236 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:39.881303 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:39.882155 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:39.882156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:39.882156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:39.882162 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:39.883748 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:39.884162 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:39.884163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:39.884163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:39.884215 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:39.884424 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:39.884425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:39.884425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:39.884586 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:39.884645 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:39.884814 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 10:07:39.884882 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252}
I20260808 10:07:39.885067 4172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:39.878711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:42393 (local address 127.2.30.129:35635)
0808 10:07:39.879135 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:39.879144 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:39.879215 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:39.879359 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:39.879360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:39.879531 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:39.879627 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:39.879628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:39.880397 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:39.880398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:39.880953 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:39.880955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:39.881335 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:39.881335 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:39.881402 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:39.881796 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:39.881800 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:39.882073 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:39.883869 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:39.883870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:39.883871 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:39.884092 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:39.884282 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:39.884283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:39.884284 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:39.884352 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:39.884643 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:39.884644 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:39.884699 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 10:07:39.884799 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 10:07:39.884920 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:07:39.914932 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:42393
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:40.001739 4175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:40.002025 4175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:40.002074 4175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:40.003360 4175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:40.003440 4175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:40.003476 4175 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 10:07:40.003504 4175 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:07:40.003547 4175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:07:40.005193 4175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.30.190:42393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:40.005523 4175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:40.005797 4175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:40.011270 4180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:40.011265 4183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:40.011270 4181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:40.011857 4175 server_base.cc:1034] running on GCE node
I20260808 10:07:40.012177 4175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:40.012431 4175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:40.013594 4175 hybrid_clock.cc:648] HybridClock initialized: now 1786183660013570 us; error 42 us; skew 500 ppm
Aug 08 10:07:40 dist-test-slave-w206 krb5kdc[3913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183660, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:40.016568 4175 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:07:40.017133 4175 webserver.cc:460] Webserver started at http://127.2.30.130:32959/ using document root <none> and password file <none>
I20260808 10:07:40.017379 4175 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:40.017449 4175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:40.017611 4175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:40.018778 4175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "203debce33054051af9850803ba3fce0"
format_stamp: "Formatted at 2026-08-08 10:07:40 on dist-test-slave-w206"
server_key: "e58490eb0b1ac5d31f2da23f7892ced6"
server_key_iv: "c771d756f3f0fc5d9932fefeb5e5ae8b"
server_key_version: "encryptionkey@0"
I20260808 10:07:40.019213 4175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "203debce33054051af9850803ba3fce0"
format_stamp: "Formatted at 2026-08-08 10:07:40 on dist-test-slave-w206"
server_key: "e58490eb0b1ac5d31f2da23f7892ced6"
server_key_iv: "c771d756f3f0fc5d9932fefeb5e5ae8b"
server_key_version: "encryptionkey@0"
I20260808 10:07:40.020752 4175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:07:40.021585 4190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:40.021812 4175 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:07:40.021922 4175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "203debce33054051af9850803ba3fce0"
format_stamp: "Formatted at 2026-08-08 10:07:40 on dist-test-slave-w206"
server_key: "e58490eb0b1ac5d31f2da23f7892ced6"
server_key_iv: "c771d756f3f0fc5d9932fefeb5e5ae8b"
server_key_version: "encryptionkey@0"
I20260808 10:07:40.022001 4175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:40.036072 4175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:40.038516 4175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:40.038707 4175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:40.039443 4175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:40.039501 4175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:40.039542 4175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:40.039561 4175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:07:40 dist-test-slave-w206 krb5kdc[3913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183660, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:40.047523 4175 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:43009
I20260808 10:07:40.047972 4175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:07:40.048514 4310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:43009 every 8 connection(s)
I20260808 10:07:40.050348 4009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.039879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:59343 (local address 127.2.30.190:42393)
0808 10:07:40.039971 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.039974 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.040257 (+ 283us) server_negotiation.cc:407] Connection header received
0808 10:07:40.041003 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.041005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.041049 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.041148 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:40.041947 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.042512 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.043778 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.044017 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.044120 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.046915 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:40.046917 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:40.046917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:40.046927 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:40.048865 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:40.049305 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:40.049306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:40.049307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:40.049371 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:40.049639 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:40.049640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:40.049640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:40.049778 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:40.049851 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.050123 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.050281 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:07:40.050362 4199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.039849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:42393 (local address 127.2.30.130:59343)
0808 10:07:40.040053 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:40.040109 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:40.040848 (+ 739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:40.041258 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:40.041263 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:40.041583 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:40.041822 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.041828 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.042625 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.042626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.043630 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.043633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.044165 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.044166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.044344 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.044962 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:40.044971 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:40.046803 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:40.048941 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:40.048946 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:40.048948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:40.049172 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:40.049450 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:40.049450 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:40.049450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:40.049556 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:40.049849 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:40.049853 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:40.049972 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 10:07:40.050057 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 10:07:40.050194 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":117,"threads_started":1}
I20260808 10:07:40.051077 4311 heartbeater.cc:347] Connected to a master server at 127.2.30.190:42393
I20260808 10:07:40.051186 4311 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:40.051383 4311 heartbeater.cc:511] Master 127.2.30.190:42393 requested a full tablet report, sending...
I20260808 10:07:40.051396 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 4175
I20260808 10:07:40.051484 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:07:40.051736 2170 external_mini_cluster.cc:1428] Setting key cfaebac12130eff93507881552b8e4fc
I20260808 10:07:40.052176 3958 ts_manager.cc:194] Registered new tserver with Master: 203debce33054051af9850803ba3fce0 (127.2.30.130:43009)
I20260808 10:07:40.052697 3958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:59343
I20260808 10:07:40.059456 4009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.052306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:35481 (local address 127.2.30.190:42393)
0808 10:07:40.052839 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.052841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.052849 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:07:40.052869 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.052869 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.052879 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.052930 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:40.053360 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.053917 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.055060 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.055318 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.055388 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.056688 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:40.056689 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:40.056690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:40.056697 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:40.058169 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:40.058530 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:40.058531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:40.058532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:40.058593 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:40.058792 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:40.058793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:40.058794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:40.058948 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:40.059060 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.059244 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.059329 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512}
I20260808 10:07:40.059464 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.052254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:42393 (local address 127.2.30.130:35481)
0808 10:07:40.052486 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:40.052497 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:40.052554 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:40.052939 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:40.052940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:40.053156 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:40.053235 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.053237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.054066 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.054067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.054945 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.054948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.055420 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.055421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.055549 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.056096 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:40.056100 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:40.056579 (+ 479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:40.058258 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:40.058259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:40.058259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:40.058442 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:40.058660 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:40.058661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:40.058661 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:40.058728 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:40.059050 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:40.059051 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:40.059103 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 10:07:40.059213 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 10:07:40.059330 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:07:40.082551 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:42393
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:40.168429 4320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:40.168691 4320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:40.168738 4320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:40.169973 4320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:40.170044 4320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:40.170083 4320 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 10:07:40.170131 4320 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:07:40.170163 4320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:07:40.171875 4320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.30.190:42393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:40.172166 4320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:40.172423 4320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:40.177682 4326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:40.177701 4329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:40.177690 4327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:40.178176 4320 server_base.cc:1034] running on GCE node
I20260808 10:07:40.178453 4320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:40.178735 4320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:40.179930 4320 hybrid_clock.cc:648] HybridClock initialized: now 1786183660179919 us; error 41 us; skew 500 ppm
Aug 08 10:07:40 dist-test-slave-w206 krb5kdc[3913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183660, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:40.182642 4320 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:07:40.183190 4320 webserver.cc:460] Webserver started at http://127.2.30.131:43345/ using document root <none> and password file <none>
I20260808 10:07:40.183385 4320 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:40.183426 4320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:40.183521 4320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:40.184644 4320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "65a8c0f690454f449ae088af9b75bb98"
format_stamp: "Formatted at 2026-08-08 10:07:40 on dist-test-slave-w206"
server_key: "70267782558e0f3a2b229d03deefb989"
server_key_iv: "86456957bf5f8204918033c24d49a6fe"
server_key_version: "encryptionkey@0"
I20260808 10:07:40.185055 4320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "65a8c0f690454f449ae088af9b75bb98"
format_stamp: "Formatted at 2026-08-08 10:07:40 on dist-test-slave-w206"
server_key: "70267782558e0f3a2b229d03deefb989"
server_key_iv: "86456957bf5f8204918033c24d49a6fe"
server_key_version: "encryptionkey@0"
I20260808 10:07:40.186728 4320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:07:40.187781 4336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:40.188006 4320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:07:40.188099 4320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "65a8c0f690454f449ae088af9b75bb98"
format_stamp: "Formatted at 2026-08-08 10:07:40 on dist-test-slave-w206"
server_key: "70267782558e0f3a2b229d03deefb989"
server_key_iv: "86456957bf5f8204918033c24d49a6fe"
server_key_version: "encryptionkey@0"
I20260808 10:07:40.188158 4320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:40.219838 4320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:40.222175 4320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:40.222365 4320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:40.222880 4320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:40.222916 4320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:40.222939 4320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:40.223038 4320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:07:40 dist-test-slave-w206 krb5kdc[3913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183660, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:40.231112 4320 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:33363
I20260808 10:07:40.231594 4320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:07:40.231419 4455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:33363 every 8 connection(s)
I20260808 10:07:40.234393 4009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.223460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:35177 (local address 127.2.30.190:42393)
0808 10:07:40.223504 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.223506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.223736 (+ 230us) server_negotiation.cc:407] Connection header received
0808 10:07:40.224268 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.224271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.224289 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.224372 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:40.225268 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.226126 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.227615 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.227986 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.228074 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.231242 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:40.231244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:40.231244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:40.231253 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:40.232931 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:40.233370 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:40.233371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:40.233372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:40.233435 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:40.233696 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:40.233697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:40.233698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:40.233844 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:40.233916 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.234211 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.234298 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:07:40.234619 4345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.223447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:42393 (local address 127.2.30.131:35177)
0808 10:07:40.223614 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:40.223652 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:40.224160 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:40.224471 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:40.224476 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:40.224861 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:40.225162 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.225169 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.226240 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.226242 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.227487 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.227492 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.228105 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.228106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.228274 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.228959 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:40.228992 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:40.231114 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:40.233030 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:40.233038 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:40.233040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:40.233283 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:40.233514 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:40.233515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:40.233515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:40.233626 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:40.233919 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:40.233927 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:40.234067 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 10:07:40.234174 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 10:07:40.234356 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
I20260808 10:07:40.236071 4456 heartbeater.cc:347] Connected to a master server at 127.2.30.190:42393
I20260808 10:07:40.236166 4456 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:40.236353 4456 heartbeater.cc:511] Master 127.2.30.190:42393 requested a full tablet report, sending...
I20260808 10:07:40.237221 3957 ts_manager.cc:194] Registered new tserver with Master: 65a8c0f690454f449ae088af9b75bb98 (127.2.30.131:33363)
I20260808 10:07:40.237775 3957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:35177
I20260808 10:07:40.239323 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 4320
I20260808 10:07:40.239410 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:07:40.239600 2170 external_mini_cluster.cc:1428] Setting key 5a0c5da87fa425100108b729f4c593a3
I20260808 10:07:40.240748 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:07:40.245503 4009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.237812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:37997 (local address 127.2.30.190:42393)
0808 10:07:40.238627 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.238629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.238639 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:07:40.238660 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.238661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.238671 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.238730 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:40.239215 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.239962 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.241022 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.241392 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.241457 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.242539 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:40.242540 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:40.242541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:40.242548 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:40.244277 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:40.244619 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:40.244620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:40.244620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:40.244674 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:40.244946 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:40.244948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:40.244948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:40.245072 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:40.245151 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.245313 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.245405 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786}
I20260808 10:07:40.245507 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.237788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:42393 (local address 127.2.30.131:37997)
0808 10:07:40.238008 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:40.238021 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:40.238077 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:40.238776 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:40.238777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:40.239059 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:40.239141 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.239144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.240067 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.240068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.240930 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.240934 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.241456 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.241457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.241597 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.242137 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:40.242142 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:40.242446 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:40.244373 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:40.244374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:40.244374 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:40.244550 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:40.244756 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:40.244757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:40.244757 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:40.244846 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:40.245142 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:40.245143 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:40.245195 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 10:07:40.245286 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 10:07:40.245403 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":96,"threads_started":1}
Aug 08 10:07:40 dist-test-slave-w206 krb5kdc[3913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:07:40 dist-test-slave-w206 krb5kdc[3913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:40.268217 4009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.260582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32820 (local address 127.2.30.190:42393)
0808 10:07:40.260670 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.260671 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.260679 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:07:40.260724 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.260725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.260737 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.260784 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:40.261508 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.262041 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.262846 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.263121 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.263189 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.265581 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:40.265583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:40.265583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:40.265591 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:40.267115 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:40.267475 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:40.267476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:40.267476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:40.267536 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:40.267689 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:40.267690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:40.267690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:40.267826 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:40.267913 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.268070 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.268142 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:07:40.270012 3957 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:07:40.271813 3957 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:07:40.279078 4475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.274429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34768 (local address 127.2.30.130:43009)
0808 10:07:40.275419 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.275423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.275440 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:07:40.275477 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.275478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.275489 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.275573 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:40.275738 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.276782 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.277939 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.278585 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.278670 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.278735 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.278905 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.278945 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":952,"thread_start_us":87,"threads_started":1}
I20260808 10:07:40.279078 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.274326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:43009 (local address 127.0.0.1:34768)
0808 10:07:40.274636 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:40.274671 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:40.274742 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:40.275585 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:40.275586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:40.275588 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:40.275646 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.275648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.276870 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.276871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.277847 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.277850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.278671 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.278672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.278767 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.278782 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 10:07:40.278885 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 10:07:40.278946 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":78,"threads_started":1}
I20260808 10:07:40.279354 4477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.274516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:32903 (local address 127.0.0.1:60216)
0808 10:07:40.274872 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:40.274881 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:40.274959 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:40.275278 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:40.275281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:40.275283 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:40.275369 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.275371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.276634 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.276635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.278017 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.278020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.279072 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.279073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.279150 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.279154 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:07:40.279208 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 10:07:40.279262 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":75,"thread_start_us":75,"threads_started":1}
I20260808 10:07:40.279675 4478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.274661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60216 (local address 127.2.30.129:32903)
0808 10:07:40.274850 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.274855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.274926 (+ 71us) server_negotiation.cc:407] Connection header received
0808 10:07:40.275164 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.275168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.275186 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.275271 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:40.275445 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.276561 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.278106 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.278875 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.278947 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.279037 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.279524 (+ 487us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.279571 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":117,"threads_started":1}
I20260808 10:07:40.279865 4476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.274445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56570 (local address 127.2.30.131:33363)
0808 10:07:40.275101 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.275106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.275115 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:07:40.275155 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.275157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.275174 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.275282 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:40.275775 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.276455 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.278105 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.278872 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.278947 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.279030 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.279612 (+ 582us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.279666 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":111,"threads_started":1}
I20260808 10:07:40.279867 4243 tablet_service.cc:1467] Processing CreateTablet for tablet dc0eaed8932d481691e245b2dc3e52df (DEFAULT_TABLE table=test-table [id=dde2f250890a4ead9463b2a7f6dd3485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:40.280084 4473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.274286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:33363 (local address 127.0.0.1:56570)
0808 10:07:40.274626 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:40.274671 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:40.274742 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:40.275645 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:40.275646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:40.275647 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:40.275683 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.275685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.276550 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.276551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.277995 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.277998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.279070 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.279072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.279314 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.279317 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:07:40.279365 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 10:07:40.280010 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":51,"thread_start_us":80,"threads_started":1}
I20260808 10:07:40.280325 4243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc0eaed8932d481691e245b2dc3e52df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:40.280324 4099 tablet_service.cc:1467] Processing CreateTablet for tablet dc0eaed8932d481691e245b2dc3e52df (DEFAULT_TABLE table=test-table [id=dde2f250890a4ead9463b2a7f6dd3485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:40.280642 4099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc0eaed8932d481691e245b2dc3e52df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:40.280676 4387 tablet_service.cc:1467] Processing CreateTablet for tablet dc0eaed8932d481691e245b2dc3e52df (DEFAULT_TABLE table=test-table [id=dde2f250890a4ead9463b2a7f6dd3485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:40.280927 4387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc0eaed8932d481691e245b2dc3e52df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:40.283802 4479 tablet_bootstrap.cc:492] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98: Bootstrap starting.
I20260808 10:07:40.284428 4480 tablet_bootstrap.cc:492] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0: Bootstrap starting.
I20260808 10:07:40.284534 4481 tablet_bootstrap.cc:492] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021: Bootstrap starting.
I20260808 10:07:40.284940 4479 tablet_bootstrap.cc:654] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:40.285140 4481 tablet_bootstrap.cc:654] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:40.285159 4480 tablet_bootstrap.cc:654] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:40.285358 4479 log.cc:824] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:40.285494 4481 log.cc:824] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:40.285521 4480 log.cc:824] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:40.286247 4479 tablet_bootstrap.cc:492] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98: No bootstrap required, opened a new log
I20260808 10:07:40.286275 4481 tablet_bootstrap.cc:492] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021: No bootstrap required, opened a new log
I20260808 10:07:40.286322 4479 ts_tablet_manager.cc:1397] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:07:40.286346 4481 ts_tablet_manager.cc:1397] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:07:40.287386 4480 tablet_bootstrap.cc:492] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0: No bootstrap required, opened a new log
I20260808 10:07:40.287528 4480 ts_tablet_manager.cc:1397] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:07:40.288455 4479 raft_consensus.cc:348] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } }
I20260808 10:07:40.288483 4481 raft_consensus.cc:348] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } }
I20260808 10:07:40.288626 4479 raft_consensus.cc:374] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:40.288636 4481 raft_consensus.cc:374] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:40.288672 4479 raft_consensus.cc:729] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65a8c0f690454f449ae088af9b75bb98, State: Initialized, Role: FOLLOWER
I20260808 10:07:40.288672 4481 raft_consensus.cc:729] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b693b00a8aee4f58b0248ad4b8a66021, State: Initialized, Role: FOLLOWER
I20260808 10:07:40.288800 4479 consensus_queue.cc:260] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } }
I20260808 10:07:40.288800 4481 consensus_queue.cc:260] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } }
I20260808 10:07:40.289117 4479 ts_tablet_manager.cc:1428] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:07:40.289116 4481 ts_tablet_manager.cc:1428] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 10:07:40.289147 4166 heartbeater.cc:503] Master 127.2.30.190:42393 was elected leader, sending a full tablet report...
I20260808 10:07:40.289229 4456 heartbeater.cc:503] Master 127.2.30.190:42393 was elected leader, sending a full tablet report...
I20260808 10:07:40.289817 4480 raft_consensus.cc:348] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } }
I20260808 10:07:40.289999 4480 raft_consensus.cc:374] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:40.290046 4480 raft_consensus.cc:729] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 203debce33054051af9850803ba3fce0, State: Initialized, Role: FOLLOWER
I20260808 10:07:40.290167 4480 consensus_queue.cc:260] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } }
I20260808 10:07:40.290457 4480 ts_tablet_manager.cc:1428] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:07:40.290483 4311 heartbeater.cc:503] Master 127.2.30.190:42393 was elected leader, sending a full tablet report...
W20260808 10:07:40.298821 4312 tablet.cc:2369] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:07:40.358362 4485 raft_consensus.cc:482] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:07:40.358494 4485 raft_consensus.cc:504] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } }
I20260808 10:07:40.358876 4485 leader_election.cc:288] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65a8c0f690454f449ae088af9b75bb98 (127.2.30.131:33363), 203debce33054051af9850803ba3fce0 (127.2.30.130:43009)
I20260808 10:07:40.362237 4476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.359210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:45141 (local address 127.2.30.131:33363)
0808 10:07:40.359254 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.359256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.359403 (+ 147us) server_negotiation.cc:407] Connection header received
0808 10:07:40.359444 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.359446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.359459 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.359507 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:40.359695 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.360325 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.361430 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.361852 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.361937 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.361947 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.362114 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.362161 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:07:40.362249 4475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.359243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:54737 (local address 127.2.30.130:43009)
0808 10:07:40.359321 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.359323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.359332 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:07:40.359365 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.359366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.359394 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.359456 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:40.359688 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.360331 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.361369 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.361854 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.361937 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.361947 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.362129 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.362177 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:07:40.362274 4056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.359114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:43009 (local address 127.2.30.129:54737)
0808 10:07:40.359178 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:40.359194 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:40.359260 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:40.359514 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:40.359515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:40.359517 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:40.359607 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.359608 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.360417 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.360418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.361297 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.361300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.361946 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.361947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.362066 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.362069 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:07:40.362122 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 10:07:40.362164 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":72}
I20260808 10:07:40.362296 4490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.359114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:33363 (local address 127.2.30.129:45141)
0808 10:07:40.359312 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:40.359324 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:40.359391 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:40.359526 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:40.359528 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:40.359530 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:40.359623 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.359625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.360409 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.360410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.361344 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.361348 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.361953 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.361954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.362037 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.362040 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:07:40.362106 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:07:40.362146 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":98,"thread_start_us":96,"threads_started":1}
I20260808 10:07:40.362623 4264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0eaed8932d481691e245b2dc3e52df" candidate_uuid: "b693b00a8aee4f58b0248ad4b8a66021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "203debce33054051af9850803ba3fce0" is_pre_election: true
I20260808 10:07:40.362655 4408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0eaed8932d481691e245b2dc3e52df" candidate_uuid: "b693b00a8aee4f58b0248ad4b8a66021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65a8c0f690454f449ae088af9b75bb98" is_pre_election: true
I20260808 10:07:40.362798 4264 raft_consensus.cc:2454] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b693b00a8aee4f58b0248ad4b8a66021 in term 0.
I20260808 10:07:40.362798 4408 raft_consensus.cc:2454] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b693b00a8aee4f58b0248ad4b8a66021 in term 0.
I20260808 10:07:40.363056 4050 leader_election.cc:302] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65a8c0f690454f449ae088af9b75bb98, b693b00a8aee4f58b0248ad4b8a66021; no voters:
I20260808 10:07:40.363266 4485 raft_consensus.cc:2781] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:07:40.363343 4485 raft_consensus.cc:482] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:07:40.363375 4485 raft_consensus.cc:3037] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:40.364140 4485 raft_consensus.cc:504] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } }
I20260808 10:07:40.364315 4485 leader_election.cc:288] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [CANDIDATE]: Term 1 election: Requested vote from peers 65a8c0f690454f449ae088af9b75bb98 (127.2.30.131:33363), 203debce33054051af9850803ba3fce0 (127.2.30.130:43009)
I20260808 10:07:40.364487 4264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0eaed8932d481691e245b2dc3e52df" candidate_uuid: "b693b00a8aee4f58b0248ad4b8a66021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "203debce33054051af9850803ba3fce0"
I20260808 10:07:40.364542 4408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc0eaed8932d481691e245b2dc3e52df" candidate_uuid: "b693b00a8aee4f58b0248ad4b8a66021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65a8c0f690454f449ae088af9b75bb98"
I20260808 10:07:40.364605 4264 raft_consensus.cc:3037] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:40.364642 4408 raft_consensus.cc:3037] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:40.365388 4408 raft_consensus.cc:2454] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b693b00a8aee4f58b0248ad4b8a66021 in term 1.
I20260808 10:07:40.365406 4264 raft_consensus.cc:2454] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b693b00a8aee4f58b0248ad4b8a66021 in term 1.
I20260808 10:07:40.365568 4050 leader_election.cc:302] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 203debce33054051af9850803ba3fce0, 65a8c0f690454f449ae088af9b75bb98, b693b00a8aee4f58b0248ad4b8a66021; no voters:
I20260808 10:07:40.365679 4485 raft_consensus.cc:2781] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:40.365860 4485 raft_consensus.cc:686] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [term 1 LEADER]: Becoming Leader. State: Replica: b693b00a8aee4f58b0248ad4b8a66021, State: Running, Role: LEADER
I20260808 10:07:40.365975 4485 consensus_queue.cc:237] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } }
I20260808 10:07:40.366885 3957 catalog_manager.cc:5495] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 reported cstate change: term changed from 0 to 1, leader changed from <none> to b693b00a8aee4f58b0248ad4b8a66021 (127.2.30.129). New cstate: current_term: 1 leader_uuid: "b693b00a8aee4f58b0248ad4b8a66021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } health_report { overall_health: HEALTHY } } }
W20260808 10:07:40.375497 4167 tablet.cc:2369] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 10:07:40 dist-test-slave-w206 krb5kdc[3913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183659, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:40.407972 4498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.400064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:42393 (local address 127.0.0.1:32826)
0808 10:07:40.400268 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:40.400278 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:40.400412 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:40.400585 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:40.400586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:40.400936 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:40.400986 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.400989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.401650 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.401651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.402178 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.402179 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.402570 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.402571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.402631 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.403284 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:40.403289 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:40.404847 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:40.406443 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:40.406450 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:40.406452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:40.406672 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:40.406924 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:40.406925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:40.406926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:40.407080 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:40.407396 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:40.407406 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:40.407470 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 10:07:40.407619 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 10:07:40.407821 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":5,"thread_start_us":97,"threads_started":1}
I20260808 10:07:40.408090 4009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.400128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32826 (local address 127.2.30.190:42393)
0808 10:07:40.400209 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.400211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.400387 (+ 176us) server_negotiation.cc:407] Connection header received
0808 10:07:40.400489 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.400490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.400503 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.400578 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:40.401080 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.401568 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.402279 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.402466 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.402570 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.404947 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:40.404949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:40.404949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:40.404956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:40.406312 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:40.406770 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:40.406771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:40.406771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:40.406828 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:40.407158 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:40.407159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:40.407159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:40.407301 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:40.407370 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.407657 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.407943 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":141,"server-negotiator.queue_time_us":49}
I20260808 10:07:40.410737 3957 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:32826:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 10:07:40.410991 3957 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:07:40.412894 4243 tablet_service.cc:1467] Processing CreateTablet for tablet 7192f6e235bb4087869264daba40bb50 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=755b7e2887864225a6be3e8066fb4f0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:07:40.412894 4099 tablet_service.cc:1467] Processing CreateTablet for tablet 7192f6e235bb4087869264daba40bb50 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=755b7e2887864225a6be3e8066fb4f0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:07:40.412948 4387 tablet_service.cc:1467] Processing CreateTablet for tablet 7192f6e235bb4087869264daba40bb50 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=755b7e2887864225a6be3e8066fb4f0f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:07:40.413103 4099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7192f6e235bb4087869264daba40bb50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:40.413103 4387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7192f6e235bb4087869264daba40bb50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:40.413105 4243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7192f6e235bb4087869264daba40bb50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:40.414395 4480 tablet_bootstrap.cc:492] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0: Bootstrap starting.
I20260808 10:07:40.414690 4481 tablet_bootstrap.cc:492] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021: Bootstrap starting.
I20260808 10:07:40.414711 4479 tablet_bootstrap.cc:492] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98: Bootstrap starting.
I20260808 10:07:40.415115 4481 tablet_bootstrap.cc:654] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:40.415314 4480 tablet_bootstrap.cc:654] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:40.415498 4479 tablet_bootstrap.cc:654] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:40.416145 4481 tablet_bootstrap.cc:492] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021: No bootstrap required, opened a new log
I20260808 10:07:40.416204 4481 ts_tablet_manager.cc:1397] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 10:07:40.416234 4479 tablet_bootstrap.cc:492] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98: No bootstrap required, opened a new log
I20260808 10:07:40.416299 4479 ts_tablet_manager.cc:1397] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 10:07:40.416319 4480 tablet_bootstrap.cc:492] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0: No bootstrap required, opened a new log
I20260808 10:07:40.416359 4480 ts_tablet_manager.cc:1397] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 10:07:40.416363 4481 raft_consensus.cc:348] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.416453 4481 raft_consensus.cc:374] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:40.416481 4481 raft_consensus.cc:729] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b693b00a8aee4f58b0248ad4b8a66021, State: Initialized, Role: FOLLOWER
I20260808 10:07:40.416472 4479 raft_consensus.cc:348] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.416522 4481 consensus_queue.cc:260] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.416518 4480 raft_consensus.cc:348] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.416563 4479 raft_consensus.cc:374] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:40.416591 4479 raft_consensus.cc:729] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65a8c0f690454f449ae088af9b75bb98, State: Initialized, Role: FOLLOWER
I20260808 10:07:40.416591 4480 raft_consensus.cc:374] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:40.416615 4480 raft_consensus.cc:729] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 203debce33054051af9850803ba3fce0, State: Initialized, Role: FOLLOWER
I20260808 10:07:40.416612 4481 ts_tablet_manager.cc:1428] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 10:07:40.416641 4479 consensus_queue.cc:260] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.416658 4480 consensus_queue.cc:260] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.416735 4479 ts_tablet_manager.cc:1428] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 10:07:40.416760 4480 ts_tablet_manager.cc:1428] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 10:07:40.416885 4489 tablet_replica.cc:439] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } }
I20260808 10:07:40.416879 4485 tablet_replica.cc:439] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } }
I20260808 10:07:40.416971 4485 tablet_replica.cc:442] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:07:40.416965 4489 tablet_replica.cc:442] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:07:40.417189 4486 tablet_replica.cc:439] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } }
I20260808 10:07:40.417320 4486 tablet_replica.cc:442] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:07:40.455188 4486 raft_consensus.cc:482] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:07:40.455323 4486 raft_consensus.cc:504] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.455709 4486 leader_election.cc:288] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b693b00a8aee4f58b0248ad4b8a66021 (127.2.30.129:32903), 203debce33054051af9850803ba3fce0 (127.2.30.130:43009)
I20260808 10:07:40.460085 4345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.455939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:32903 (local address 127.2.30.131:40491)
0808 10:07:40.456051 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:40.456066 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:40.456134 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:40.456378 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:40.456379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:40.456381 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:40.456499 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.456501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.457482 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.457482 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.458858 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.458862 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.459723 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.459725 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.459862 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.459866 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:07:40.459940 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 10:07:40.459990 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":33}
I20260808 10:07:40.460147 4475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.456250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:56433 (local address 127.2.30.130:43009)
0808 10:07:40.456354 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.456356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.456368 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:07:40.456398 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.456399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.456408 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.456463 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:40.456648 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.457360 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.459005 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.459558 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.459637 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.459648 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.459911 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.459995 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 10:07:40.460395 4502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.456072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:43009 (local address 127.2.30.131:56433)
0808 10:07:40.456272 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:40.456282 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:40.456344 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:40.456490 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:40.456490 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:40.456492 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:40.456561 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.456563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.457472 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.457474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.458858 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.458862 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.459683 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.459684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.459783 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.459789 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:07:40.459871 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:07:40.459950 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":119,"thread_start_us":134,"threads_started":1}
I20260808 10:07:40.460403 4478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.456064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:40491 (local address 127.2.30.129:32903)
0808 10:07:40.456142 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.456145 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.456160 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:07:40.456200 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.456201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.456215 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.456354 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:40.456588 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.457368 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.459033 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.459636 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.459699 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.459708 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.460290 (+ 582us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.460342 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:07:40.460727 4119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7192f6e235bb4087869264daba40bb50" candidate_uuid: "65a8c0f690454f449ae088af9b75bb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b693b00a8aee4f58b0248ad4b8a66021" is_pre_election: true
I20260808 10:07:40.460762 4264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7192f6e235bb4087869264daba40bb50" candidate_uuid: "65a8c0f690454f449ae088af9b75bb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "203debce33054051af9850803ba3fce0" is_pre_election: true
I20260808 10:07:40.460865 4264 raft_consensus.cc:2454] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65a8c0f690454f449ae088af9b75bb98 in term 0.
I20260808 10:07:40.460866 4119 raft_consensus.cc:2454] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65a8c0f690454f449ae088af9b75bb98 in term 0.
I20260808 10:07:40.461109 4340 leader_election.cc:302] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 203debce33054051af9850803ba3fce0, 65a8c0f690454f449ae088af9b75bb98; no voters:
I20260808 10:07:40.461308 4486 raft_consensus.cc:2781] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:07:40.461357 4486 raft_consensus.cc:482] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:07:40.461380 4486 raft_consensus.cc:3037] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:40.462186 4486 raft_consensus.cc:504] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.462344 4486 leader_election.cc:288] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [CANDIDATE]: Term 1 election: Requested vote from peers b693b00a8aee4f58b0248ad4b8a66021 (127.2.30.129:32903), 203debce33054051af9850803ba3fce0 (127.2.30.130:43009)
I20260808 10:07:40.462563 4119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7192f6e235bb4087869264daba40bb50" candidate_uuid: "65a8c0f690454f449ae088af9b75bb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b693b00a8aee4f58b0248ad4b8a66021"
I20260808 10:07:40.462560 4264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7192f6e235bb4087869264daba40bb50" candidate_uuid: "65a8c0f690454f449ae088af9b75bb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "203debce33054051af9850803ba3fce0"
I20260808 10:07:40.462646 4119 raft_consensus.cc:3037] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:40.462652 4264 raft_consensus.cc:3037] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:40.463460 4264 raft_consensus.cc:2454] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65a8c0f690454f449ae088af9b75bb98 in term 1.
I20260808 10:07:40.463460 4119 raft_consensus.cc:2454] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65a8c0f690454f449ae088af9b75bb98 in term 1.
I20260808 10:07:40.463871 4340 leader_election.cc:302] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 203debce33054051af9850803ba3fce0, 65a8c0f690454f449ae088af9b75bb98; no voters:
I20260808 10:07:40.464000 4486 raft_consensus.cc:2781] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:40.464151 4486 raft_consensus.cc:686] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [term 1 LEADER]: Becoming Leader. State: Replica: 65a8c0f690454f449ae088af9b75bb98, State: Running, Role: LEADER
I20260808 10:07:40.464290 4486 consensus_queue.cc:237] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.464847 4503 tablet_replica.cc:439] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98: TxnStatusTablet state changed. Reason: New leader 65a8c0f690454f449ae088af9b75bb98. Latest consensus state: current_term: 1 leader_uuid: "65a8c0f690454f449ae088af9b75bb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } }
I20260808 10:07:40.464936 4503 tablet_replica.cc:442] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:07:40.465126 4506 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 10:07:40.465245 4506 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 10:07:40.465126 3956 catalog_manager.cc:5495] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 65a8c0f690454f449ae088af9b75bb98 (127.2.30.131). New cstate: current_term: 1 leader_uuid: "65a8c0f690454f449ae088af9b75bb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } health_report { overall_health: HEALTHY } } }
I20260808 10:07:40.481581 4476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.478637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56580 (local address 127.2.30.131:33363)
0808 10:07:40.478705 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.478707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.478747 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:07:40.478865 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.478866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.478887 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.478955 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:40.479152 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.479655 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.480770 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.480961 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.481019 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.481237 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:40.481337 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:40.481396 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.481477 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.481496 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:07:40.481621 4498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.478549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:33363 (local address 127.0.0.1:56580)
0808 10:07:40.478681 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:40.478695 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:40.478764 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:40.479015 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:40.479016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:40.479017 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:07:40.479077 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.479080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.479738 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.479739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.480661 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.480664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.481043 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.481044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.481115 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.481135 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:07:40.481404 (+ 269us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:07:40.481412 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:07:40.481476 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 10:07:40.481529 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 10:07:40.483796 4264 raft_consensus.cc:1261] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Refusing update from remote peer 65a8c0f690454f449ae088af9b75bb98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:40.483819 4119 raft_consensus.cc:1261] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021 [term 1 FOLLOWER]: Refusing update from remote peer 65a8c0f690454f449ae088af9b75bb98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:40.484158 4503 consensus_queue.cc:1035] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:40.484380 4486 consensus_queue.cc:1035] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:40.485756 4485 tablet_replica.cc:439] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021: TxnStatusTablet state changed. Reason: New leader 65a8c0f690454f449ae088af9b75bb98. Latest consensus state: current_term: 1 leader_uuid: "65a8c0f690454f449ae088af9b75bb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } }
I20260808 10:07:40.485857 4485 tablet_replica.cc:442] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 10:07:40.485978 4457 tablet.cc:2369] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:07:40.486099 4489 tablet_replica.cc:439] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0: TxnStatusTablet state changed. Reason: New leader 65a8c0f690454f449ae088af9b75bb98. Latest consensus state: current_term: 1 leader_uuid: "65a8c0f690454f449ae088af9b75bb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } }
I20260808 10:07:40.486196 4489 tablet_replica.cc:442] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:07:40.487566 4486 tablet_replica.cc:439] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65a8c0f690454f449ae088af9b75bb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } }
I20260808 10:07:40.487672 4486 tablet_replica.cc:442] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:07:40.487747 4486 tablet_replica.cc:439] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "65a8c0f690454f449ae088af9b75bb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } }
I20260808 10:07:40.487804 4486 tablet_replica.cc:442] T 7192f6e235bb4087869264daba40bb50 P 65a8c0f690454f449ae088af9b75bb98: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:07:40.487784 4510 mvcc.cc:205] Tried to move back new op lower bound from 7316208273338589184 to 7316208273262448640. Current Snapshot: MvccSnapshot[applied={T|T < 7316208273338589184}]
I20260808 10:07:40.487782 4485 tablet_replica.cc:439] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65a8c0f690454f449ae088af9b75bb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } }
I20260808 10:07:40.487869 4485 tablet_replica.cc:442] T 7192f6e235bb4087869264daba40bb50 P b693b00a8aee4f58b0248ad4b8a66021: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:07:40.487962 4489 tablet_replica.cc:439] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "65a8c0f690454f449ae088af9b75bb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } } }
I20260808 10:07:40.488044 4489 tablet_replica.cc:442] T 7192f6e235bb4087869264daba40bb50 P 203debce33054051af9850803ba3fce0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:07:40.488158 4509 mvcc.cc:205] Tried to move back new op lower bound from 7316208273338589184 to 7316208273262448640. Current Snapshot: MvccSnapshot[applied={T|T < 7316208273338589184}]
I20260808 10:07:40.488823 4511 mvcc.cc:205] Tried to move back new op lower bound from 7316208273338589184 to 7316208273262448640. Current Snapshot: MvccSnapshot[applied={T|T < 7316208273338589184 or (T in {7316208273338589184})}]
I20260808 10:07:40.493675 4478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.490768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60222 (local address 127.2.30.129:32903)
0808 10:07:40.490817 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.490820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.490831 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:07:40.490920 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.490921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.490940 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.491047 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:40.491230 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.491914 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.492774 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.493016 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.493106 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.493298 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:40.493416 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:40.493520 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.493563 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.493588 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:07:40.499327 4476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.496518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:59933 (local address 127.2.30.131:33363)
0808 10:07:40.496566 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.496568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.496679 (+ 111us) server_negotiation.cc:407] Connection header received
0808 10:07:40.496752 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.496752 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.496761 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.496802 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:40.496940 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.497384 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.498537 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.498741 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.498798 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.499080 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:40.499108 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:40.499148 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.499250 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.499271 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:07:40.499495 4521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.496423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:33363 (local address 127.2.30.129:59933)
0808 10:07:40.496593 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:40.496605 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:40.496682 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:40.496806 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:40.496807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:40.496808 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:07:40.496877 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.496878 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.497458 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.497459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.498446 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.498449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.498825 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.498826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.498898 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.498916 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:07:40.499174 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:07:40.499176 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:07:40.499229 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 10:07:40.499277 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260808 10:07:40.502610 4264 raft_consensus.cc:1261] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Refusing update from remote peer b693b00a8aee4f58b0248ad4b8a66021: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:40.502651 4408 raft_consensus.cc:1261] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Refusing update from remote peer b693b00a8aee4f58b0248ad4b8a66021: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:40.502959 4485 consensus_queue.cc:1035] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [LEADER]: Connected to new peer: Peer: permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:40.503146 4491 consensus_queue.cc:1035] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:40.512528 4462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.509720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:32903 (local address 127.2.30.131:58811)
0808 10:07:40.509876 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:40.509888 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:40.509949 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:40.510098 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:40.510099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:40.510101 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:07:40.510173 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.510174 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.510829 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.510830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.511486 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:40.511488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.511983 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.511984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:40.512046 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.512061 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:07:40.512350 (+ 289us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:07:40.512352 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:07:40.512403 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 10:07:40.512460 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260808 10:07:40.512528 4478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:40.509784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:58811 (local address 127.2.30.129:32903)
0808 10:07:40.509821 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:40.509824 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:40.509941 (+ 117us) server_negotiation.cc:407] Connection header received
0808 10:07:40.510013 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:40.510014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:40.510023 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:40.510093 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:40.510249 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.510740 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.511597 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:40.511901 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:40.511976 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:40.512158 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:40.512195 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:40.512284 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:40.512424 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 10:07:40.512448 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 10:07:40.523188 3958 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:07:40.523329 3958 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:07:40.524263 3958 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b: Sending DeleteTablet for 3 replicas of tablet dc0eaed8932d481691e245b2dc3e52df
I20260808 10:07:40.524626 4243 tablet_service.cc:1514] Processing DeleteTablet for tablet dc0eaed8932d481691e245b2dc3e52df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:40 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:34768
I20260808 10:07:40.524680 4099 tablet_service.cc:1514] Processing DeleteTablet for tablet dc0eaed8932d481691e245b2dc3e52df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:40 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:60216
I20260808 10:07:40.525086 3958 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32820:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:07:40.525251 3958 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:07:40.525663 4387 tablet_service.cc:1514] Processing DeleteTablet for tablet dc0eaed8932d481691e245b2dc3e52df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:40 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:56570
I20260808 10:07:40.526006 4531 tablet_replica.cc:330] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98: stopping tablet replica
I20260808 10:07:40.526212 4531 raft_consensus.cc:2229] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:07:40.526335 4531 raft_consensus.cc:2258] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:40.526734 4243 tablet_service.cc:1467] Processing CreateTablet for tablet 9b3a344cb7a64e21b9c83bee21015c47 (DEFAULT_TABLE table=test-table [id=6d493b5fe53049c0b2b0f9e56d3e3622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:40.526793 4099 tablet_service.cc:1467] Processing CreateTablet for tablet 9b3a344cb7a64e21b9c83bee21015c47 (DEFAULT_TABLE table=test-table [id=6d493b5fe53049c0b2b0f9e56d3e3622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:40.526878 4243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b3a344cb7a64e21b9c83bee21015c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:40.526906 4099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b3a344cb7a64e21b9c83bee21015c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:40.527276 4387 tablet_service.cc:1467] Processing CreateTablet for tablet 9b3a344cb7a64e21b9c83bee21015c47 (DEFAULT_TABLE table=test-table [id=6d493b5fe53049c0b2b0f9e56d3e3622]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:40.527410 4387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b3a344cb7a64e21b9c83bee21015c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:40.528342 4480 tablet_bootstrap.cc:492] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0: Bootstrap starting.
I20260808 10:07:40.528376 4481 tablet_bootstrap.cc:492] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021: Bootstrap starting.
I20260808 10:07:40.528797 4480 tablet_bootstrap.cc:654] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:40.528797 4481 tablet_bootstrap.cc:654] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:40.528802 4479 tablet_bootstrap.cc:492] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98: Bootstrap starting.
I20260808 10:07:40.529246 4479 tablet_bootstrap.cc:654] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:40.529601 4531 ts_tablet_manager.cc:1905] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:40.531426 4531 ts_tablet_manager.cc:1918] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:07:40.531571 4480 tablet_bootstrap.cc:492] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0: No bootstrap required, opened a new log
I20260808 10:07:40.531631 4480 ts_tablet_manager.cc:1397] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 10:07:40.531585 4531 log.cc:1196] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/dc0eaed8932d481691e245b2dc3e52df
I20260808 10:07:40.531800 4480 raft_consensus.cc:348] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.531898 4480 raft_consensus.cc:374] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:40.531929 4480 raft_consensus.cc:729] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 203debce33054051af9850803ba3fce0, State: Initialized, Role: FOLLOWER
I20260808 10:07:40.531972 4480 consensus_queue.cc:260] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.532066 4480 ts_tablet_manager.cc:1428] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 10:07:40.532155 4481 tablet_bootstrap.cc:492] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021: No bootstrap required, opened a new log
I20260808 10:07:40.532199 4481 ts_tablet_manager.cc:1397] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 10:07:40.532290 4479 tablet_bootstrap.cc:492] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98: No bootstrap required, opened a new log
I20260808 10:07:40.532339 4479 ts_tablet_manager.cc:1397] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 10:07:40.532346 4481 raft_consensus.cc:348] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.532416 4481 raft_consensus.cc:374] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:40.532436 4481 raft_consensus.cc:729] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b693b00a8aee4f58b0248ad4b8a66021, State: Initialized, Role: FOLLOWER
I20260808 10:07:40.532475 4481 consensus_queue.cc:260] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.532495 4479 raft_consensus.cc:348] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.532565 4479 raft_consensus.cc:374] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:40.532577 4481 ts_tablet_manager.cc:1428] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 10:07:40.532588 4479 raft_consensus.cc:729] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65a8c0f690454f449ae088af9b75bb98, State: Initialized, Role: FOLLOWER
I20260808 10:07:40.532637 4479 consensus_queue.cc:260] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.532738 4479 ts_tablet_manager.cc:1428] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 10:07:40.533552 4531 ts_tablet_manager.cc:1939] T dc0eaed8932d481691e245b2dc3e52df P 65a8c0f690454f449ae088af9b75bb98: Deleting consensus metadata
I20260808 10:07:40.534108 3944 catalog_manager.cc:4841] TS 65a8c0f690454f449ae088af9b75bb98 (127.2.30.131:33363): tablet dc0eaed8932d481691e245b2dc3e52df (table test-table [id=dde2f250890a4ead9463b2a7f6dd3485]) successfully deleted
I20260808 10:07:40.535121 4529 tablet_replica.cc:330] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0: stopping tablet replica
I20260808 10:07:40.535322 4530 tablet_replica.cc:330] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021: stopping tablet replica
I20260808 10:07:40.535533 4529 raft_consensus.cc:2229] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:07:40.535687 4530 raft_consensus.cc:2229] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:07:40.535812 4529 raft_consensus.cc:2258] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:40.535981 4530 raft_consensus.cc:2258] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:40.536728 4530 ts_tablet_manager.cc:1905] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:40.537082 4529 ts_tablet_manager.cc:1905] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:40.538594 4530 ts_tablet_manager.cc:1918] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:07:40.538722 4530 log.cc:1196] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/wal/wals/dc0eaed8932d481691e245b2dc3e52df
I20260808 10:07:40.538807 4529 ts_tablet_manager.cc:1918] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:07:40.538893 4529 log.cc:1196] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/dc0eaed8932d481691e245b2dc3e52df
I20260808 10:07:40.539176 4530 ts_tablet_manager.cc:1939] T dc0eaed8932d481691e245b2dc3e52df P b693b00a8aee4f58b0248ad4b8a66021: Deleting consensus metadata
I20260808 10:07:40.539227 4529 ts_tablet_manager.cc:1939] T dc0eaed8932d481691e245b2dc3e52df P 203debce33054051af9850803ba3fce0: Deleting consensus metadata
I20260808 10:07:40.539676 3945 catalog_manager.cc:4841] TS 203debce33054051af9850803ba3fce0 (127.2.30.130:43009): tablet dc0eaed8932d481691e245b2dc3e52df (table test-table [id=dde2f250890a4ead9463b2a7f6dd3485]) successfully deleted
I20260808 10:07:40.539760 3945 catalog_manager.cc:4841] TS b693b00a8aee4f58b0248ad4b8a66021 (127.2.30.129:32903): tablet dc0eaed8932d481691e245b2dc3e52df (table test-table [id=dde2f250890a4ead9463b2a7f6dd3485]) successfully deleted
I20260808 10:07:40.686260 4485 raft_consensus.cc:482] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:07:40.686373 4485 raft_consensus.cc:504] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.686578 4485 leader_election.cc:288] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 203debce33054051af9850803ba3fce0 (127.2.30.130:43009), 65a8c0f690454f449ae088af9b75bb98 (127.2.30.131:33363)
I20260808 10:07:40.686842 4408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3a344cb7a64e21b9c83bee21015c47" candidate_uuid: "b693b00a8aee4f58b0248ad4b8a66021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65a8c0f690454f449ae088af9b75bb98" is_pre_election: true
I20260808 10:07:40.686942 4408 raft_consensus.cc:2454] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b693b00a8aee4f58b0248ad4b8a66021 in term 0.
I20260808 10:07:40.686889 4264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3a344cb7a64e21b9c83bee21015c47" candidate_uuid: "b693b00a8aee4f58b0248ad4b8a66021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "203debce33054051af9850803ba3fce0" is_pre_election: true
I20260808 10:07:40.687019 4264 raft_consensus.cc:2454] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b693b00a8aee4f58b0248ad4b8a66021 in term 0.
I20260808 10:07:40.687175 4050 leader_election.cc:302] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65a8c0f690454f449ae088af9b75bb98, b693b00a8aee4f58b0248ad4b8a66021; no voters:
I20260808 10:07:40.687316 4485 raft_consensus.cc:2781] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:07:40.687359 4485 raft_consensus.cc:482] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:07:40.687412 4485 raft_consensus.cc:3037] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:40.688226 4485 raft_consensus.cc:504] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.688442 4485 leader_election.cc:288] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [CANDIDATE]: Term 1 election: Requested vote from peers 203debce33054051af9850803ba3fce0 (127.2.30.130:43009), 65a8c0f690454f449ae088af9b75bb98 (127.2.30.131:33363)
I20260808 10:07:40.688659 4264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3a344cb7a64e21b9c83bee21015c47" candidate_uuid: "b693b00a8aee4f58b0248ad4b8a66021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "203debce33054051af9850803ba3fce0"
I20260808 10:07:40.688683 4408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b3a344cb7a64e21b9c83bee21015c47" candidate_uuid: "b693b00a8aee4f58b0248ad4b8a66021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65a8c0f690454f449ae088af9b75bb98"
I20260808 10:07:40.688763 4264 raft_consensus.cc:3037] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:40.688764 4408 raft_consensus.cc:3037] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:40.689399 4408 raft_consensus.cc:2454] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b693b00a8aee4f58b0248ad4b8a66021 in term 1.
I20260808 10:07:40.689503 4264 raft_consensus.cc:2454] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b693b00a8aee4f58b0248ad4b8a66021 in term 1.
I20260808 10:07:40.689592 4050 leader_election.cc:302] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65a8c0f690454f449ae088af9b75bb98, b693b00a8aee4f58b0248ad4b8a66021; no voters:
I20260808 10:07:40.689692 4485 raft_consensus.cc:2781] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:40.689754 4485 raft_consensus.cc:686] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [term 1 LEADER]: Becoming Leader. State: Replica: b693b00a8aee4f58b0248ad4b8a66021, State: Running, Role: LEADER
I20260808 10:07:40.689832 4485 consensus_queue.cc:237] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:40.690451 3958 catalog_manager.cc:5495] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 reported cstate change: term changed from 0 to 1, leader changed from <none> to b693b00a8aee4f58b0248ad4b8a66021 (127.2.30.129). New cstate: current_term: 1 leader_uuid: "b693b00a8aee4f58b0248ad4b8a66021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } health_report { overall_health: UNKNOWN } } }
I20260808 10:07:40.717154 4264 raft_consensus.cc:1261] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Refusing update from remote peer b693b00a8aee4f58b0248ad4b8a66021: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:40.717154 4408 raft_consensus.cc:1261] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Refusing update from remote peer b693b00a8aee4f58b0248ad4b8a66021: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:40.717449 4485 consensus_queue.cc:1035] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [LEADER]: Connected to new peer: Peer: permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:40.717579 4491 consensus_queue.cc:1035] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:07:40.723083 4511 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 10:07:42.248769 3958 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:07:42.248879 3958 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32820:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:07:42.249886 3958 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b: Sending DeleteTablet for 3 replicas of tablet 9b3a344cb7a64e21b9c83bee21015c47
I20260808 10:07:42.250161 4099 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b3a344cb7a64e21b9c83bee21015c47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:42 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:60216
I20260808 10:07:42.250188 4387 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b3a344cb7a64e21b9c83bee21015c47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:42 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:56570
I20260808 10:07:42.250257 4243 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b3a344cb7a64e21b9c83bee21015c47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:42 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:34768
I20260808 10:07:42.250455 4563 tablet_replica.cc:330] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021: stopping tablet replica
I20260808 10:07:42.250522 4563 raft_consensus.cc:2229] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:07:42.250600 4563 raft_consensus.cc:2258] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:42.250710 4564 tablet_replica.cc:330] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98: stopping tablet replica
I20260808 10:07:42.250828 4564 raft_consensus.cc:2229] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:07:42.250880 4564 raft_consensus.cc:2258] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:42.251034 4566 tablet_replica.cc:330] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0: stopping tablet replica
I20260808 10:07:42.251101 4566 raft_consensus.cc:2229] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:07:42.251147 4566 raft_consensus.cc:2258] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:42.251260 4564 ts_tablet_manager.cc:1905] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:42.252625 4563 ts_tablet_manager.cc:1905] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:42.253131 4564 ts_tablet_manager.cc:1918] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:07:42.253196 4564 log.cc:1196] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/9b3a344cb7a64e21b9c83bee21015c47
I20260808 10:07:42.253444 4564 ts_tablet_manager.cc:1939] T 9b3a344cb7a64e21b9c83bee21015c47 P 65a8c0f690454f449ae088af9b75bb98: Deleting consensus metadata
I20260808 10:07:42.253582 4566 ts_tablet_manager.cc:1905] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:42.253928 3944 catalog_manager.cc:4841] TS 65a8c0f690454f449ae088af9b75bb98 (127.2.30.131:33363): tablet 9b3a344cb7a64e21b9c83bee21015c47 (table test-table [id=6d493b5fe53049c0b2b0f9e56d3e3622]) successfully deleted
I20260808 10:07:42.254716 4563 ts_tablet_manager.cc:1918] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:07:42.254789 4563 log.cc:1196] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/wal/wals/9b3a344cb7a64e21b9c83bee21015c47
I20260808 10:07:42.255084 4563 ts_tablet_manager.cc:1939] T 9b3a344cb7a64e21b9c83bee21015c47 P b693b00a8aee4f58b0248ad4b8a66021: Deleting consensus metadata
I20260808 10:07:42.255399 4566 ts_tablet_manager.cc:1918] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:07:42.255461 4566 log.cc:1196] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/9b3a344cb7a64e21b9c83bee21015c47
I20260808 10:07:42.255462 3945 catalog_manager.cc:4841] TS b693b00a8aee4f58b0248ad4b8a66021 (127.2.30.129:32903): tablet 9b3a344cb7a64e21b9c83bee21015c47 (table test-table [id=6d493b5fe53049c0b2b0f9e56d3e3622]) successfully deleted
I20260808 10:07:42.255679 4566 ts_tablet_manager.cc:1939] T 9b3a344cb7a64e21b9c83bee21015c47 P 203debce33054051af9850803ba3fce0: Deleting consensus metadata
I20260808 10:07:42.256042 3945 catalog_manager.cc:4841] TS 203debce33054051af9850803ba3fce0 (127.2.30.130:43009): tablet 9b3a344cb7a64e21b9c83bee21015c47 (table test-table [id=6d493b5fe53049c0b2b0f9e56d3e3622]) successfully deleted
Aug 08 10:07:42 dist-test-slave-w206 krb5kdc[3913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183662, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:07:42 dist-test-slave-w206 krb5kdc[3913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183662, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:42.277588 4574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:42.269590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32830 (local address 127.2.30.190:42393)
0808 10:07:42.269775 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:42.269777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:42.269788 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:07:42.269863 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:42.269863 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:42.269875 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:42.269938 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:42.270508 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:42.271152 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:42.271973 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:42.272249 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:42.272309 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:42.274659 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:42.274661 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:42.274662 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:42.274670 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:42.276304 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:42.276644 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:42.276645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:42.276646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:42.276711 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:42.276954 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:42.276955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:42.276956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:42.277162 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:42.277293 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:42.277370 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 10:07:42.277446 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":116,"threads_started":1}
I20260808 10:07:42.279367 3958 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:07:42.279520 3958 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:07:42.281059 4099 tablet_service.cc:1467] Processing CreateTablet for tablet 8459b86c68bc4c07bbde48afb822961b (DEFAULT_TABLE table=test-table [id=77f123dfa86547558e2585d7ce47c069]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:42.281071 4243 tablet_service.cc:1467] Processing CreateTablet for tablet 8459b86c68bc4c07bbde48afb822961b (DEFAULT_TABLE table=test-table [id=77f123dfa86547558e2585d7ce47c069]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:42.281217 4243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8459b86c68bc4c07bbde48afb822961b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:42.281217 4099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8459b86c68bc4c07bbde48afb822961b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:42.282316 4575 tablet_bootstrap.cc:492] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021: Bootstrap starting.
I20260808 10:07:42.282388 4387 tablet_service.cc:1467] Processing CreateTablet for tablet 8459b86c68bc4c07bbde48afb822961b (DEFAULT_TABLE table=test-table [id=77f123dfa86547558e2585d7ce47c069]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:42.282478 4576 tablet_bootstrap.cc:492] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0: Bootstrap starting.
I20260808 10:07:42.282521 4387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8459b86c68bc4c07bbde48afb822961b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:42.282802 4575 tablet_bootstrap.cc:654] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:42.282910 4576 tablet_bootstrap.cc:654] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:42.283528 4575 tablet_bootstrap.cc:492] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021: No bootstrap required, opened a new log
I20260808 10:07:42.283594 4576 tablet_bootstrap.cc:492] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0: No bootstrap required, opened a new log
I20260808 10:07:42.283588 4575 ts_tablet_manager.cc:1397] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:42.283632 4576 ts_tablet_manager.cc:1397] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:42.283711 4575 raft_consensus.cc:348] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.283746 4576 raft_consensus.cc:348] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.283809 4575 raft_consensus.cc:374] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:42.283810 4576 raft_consensus.cc:374] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:42.283833 4575 raft_consensus.cc:729] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b693b00a8aee4f58b0248ad4b8a66021, State: Initialized, Role: FOLLOWER
I20260808 10:07:42.283833 4576 raft_consensus.cc:729] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 203debce33054051af9850803ba3fce0, State: Initialized, Role: FOLLOWER
I20260808 10:07:42.283880 4575 consensus_queue.cc:260] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.283896 4576 consensus_queue.cc:260] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.283984 4575 ts_tablet_manager.cc:1428] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:07:42.284068 4576 ts_tablet_manager.cc:1428] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:07:42.284284 4580 tablet_bootstrap.cc:492] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98: Bootstrap starting.
I20260808 10:07:42.284755 4580 tablet_bootstrap.cc:654] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:42.285360 4580 tablet_bootstrap.cc:492] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98: No bootstrap required, opened a new log
I20260808 10:07:42.285405 4580 ts_tablet_manager.cc:1397] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:42.285552 4580 raft_consensus.cc:348] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.285629 4580 raft_consensus.cc:374] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:42.285667 4580 raft_consensus.cc:729] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65a8c0f690454f449ae088af9b75bb98, State: Initialized, Role: FOLLOWER
I20260808 10:07:42.285722 4580 consensus_queue.cc:260] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.285846 4580 ts_tablet_manager.cc:1428] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:42.303357 4539 raft_consensus.cc:482] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:07:42.303467 4539 raft_consensus.cc:504] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.303644 4539 leader_election.cc:288] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 203debce33054051af9850803ba3fce0 (127.2.30.130:43009), 65a8c0f690454f449ae088af9b75bb98 (127.2.30.131:33363)
I20260808 10:07:42.303896 4264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8459b86c68bc4c07bbde48afb822961b" candidate_uuid: "b693b00a8aee4f58b0248ad4b8a66021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "203debce33054051af9850803ba3fce0" is_pre_election: true
I20260808 10:07:42.303908 4408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8459b86c68bc4c07bbde48afb822961b" candidate_uuid: "b693b00a8aee4f58b0248ad4b8a66021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65a8c0f690454f449ae088af9b75bb98" is_pre_election: true
I20260808 10:07:42.304005 4408 raft_consensus.cc:2454] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b693b00a8aee4f58b0248ad4b8a66021 in term 0.
I20260808 10:07:42.304005 4264 raft_consensus.cc:2454] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b693b00a8aee4f58b0248ad4b8a66021 in term 0.
I20260808 10:07:42.304176 4050 leader_election.cc:302] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 203debce33054051af9850803ba3fce0, 65a8c0f690454f449ae088af9b75bb98, b693b00a8aee4f58b0248ad4b8a66021; no voters:
I20260808 10:07:42.304301 4539 raft_consensus.cc:2781] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:07:42.304340 4539 raft_consensus.cc:482] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:07:42.304387 4539 raft_consensus.cc:3037] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:42.305368 4539 raft_consensus.cc:504] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.305534 4539 leader_election.cc:288] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [CANDIDATE]: Term 1 election: Requested vote from peers 203debce33054051af9850803ba3fce0 (127.2.30.130:43009), 65a8c0f690454f449ae088af9b75bb98 (127.2.30.131:33363)
I20260808 10:07:42.305747 4408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8459b86c68bc4c07bbde48afb822961b" candidate_uuid: "b693b00a8aee4f58b0248ad4b8a66021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65a8c0f690454f449ae088af9b75bb98"
I20260808 10:07:42.305843 4408 raft_consensus.cc:3037] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:42.305884 4264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8459b86c68bc4c07bbde48afb822961b" candidate_uuid: "b693b00a8aee4f58b0248ad4b8a66021" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "203debce33054051af9850803ba3fce0"
I20260808 10:07:42.306006 4264 raft_consensus.cc:3037] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:42.306519 4408 raft_consensus.cc:2454] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b693b00a8aee4f58b0248ad4b8a66021 in term 1.
I20260808 10:07:42.306674 4264 raft_consensus.cc:2454] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b693b00a8aee4f58b0248ad4b8a66021 in term 1.
I20260808 10:07:42.306700 4050 leader_election.cc:302] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65a8c0f690454f449ae088af9b75bb98, b693b00a8aee4f58b0248ad4b8a66021; no voters:
I20260808 10:07:42.306915 4539 raft_consensus.cc:2781] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:42.306967 4539 raft_consensus.cc:686] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [term 1 LEADER]: Becoming Leader. State: Replica: b693b00a8aee4f58b0248ad4b8a66021, State: Running, Role: LEADER
I20260808 10:07:42.307080 4539 consensus_queue.cc:237] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.307742 3956 catalog_manager.cc:5495] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 reported cstate change: term changed from 0 to 1, leader changed from <none> to b693b00a8aee4f58b0248ad4b8a66021 (127.2.30.129). New cstate: current_term: 1 leader_uuid: "b693b00a8aee4f58b0248ad4b8a66021" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } health_report { overall_health: UNKNOWN } } }
I20260808 10:07:42.316368 4584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:42.313958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60238 (local address 127.2.30.129:32903)
0808 10:07:42.314125 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:42.314127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:42.314134 (+ 7us) server_negotiation.cc:407] Connection header received
0808 10:07:42.314154 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:42.314155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:42.314165 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:42.314226 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:42.314368 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:42.314846 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:42.315671 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:42.315871 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:42.315933 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:42.316060 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:42.316089 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:42.316128 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:42.316245 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 10:07:42.316266 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":112,"threads_started":1}
I20260808 10:07:42.318897 4264 raft_consensus.cc:1261] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Refusing update from remote peer b693b00a8aee4f58b0248ad4b8a66021: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:42.318881 4408 raft_consensus.cc:1261] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Refusing update from remote peer b693b00a8aee4f58b0248ad4b8a66021: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:42.319288 4558 consensus_queue.cc:1035] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [LEADER]: Connected to new peer: Peer: permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:42.319562 4558 consensus_queue.cc:1035] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:42.329830 3956 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:07:42.329943 3956 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:07:42.330566 3956 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b: Sending DeleteTablet for 3 replicas of tablet 8459b86c68bc4c07bbde48afb822961b
I20260808 10:07:42.330829 4099 tablet_service.cc:1514] Processing DeleteTablet for tablet 8459b86c68bc4c07bbde48afb822961b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:42 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:60216
I20260808 10:07:42.330871 4387 tablet_service.cc:1514] Processing DeleteTablet for tablet 8459b86c68bc4c07bbde48afb822961b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:42 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:56570
I20260808 10:07:42.331037 4563 tablet_replica.cc:330] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021: stopping tablet replica
I20260808 10:07:42.331035 4564 tablet_replica.cc:330] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98: stopping tablet replica
I20260808 10:07:42.331070 4243 tablet_service.cc:1514] Processing DeleteTablet for tablet 8459b86c68bc4c07bbde48afb822961b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:42 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:34768
I20260808 10:07:42.331105 4563 raft_consensus.cc:2229] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:07:42.331111 4564 raft_consensus.cc:2229] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:07:42.331151 4566 tablet_replica.cc:330] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0: stopping tablet replica
I20260808 10:07:42.331163 4563 raft_consensus.cc:2258] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:42.331171 4564 raft_consensus.cc:2258] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:42.331192 4566 raft_consensus.cc:2229] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:07:42.331231 4566 raft_consensus.cc:2258] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:42.331583 4564 ts_tablet_manager.cc:1905] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:42.331491 3956 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:07:42.331672 3956 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:07:42.331771 4566 ts_tablet_manager.cc:1905] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:42.333510 4563 ts_tablet_manager.cc:1905] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:42.333837 4564 ts_tablet_manager.cc:1918] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:07:42.333925 4564 log.cc:1196] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/8459b86c68bc4c07bbde48afb822961b
I20260808 10:07:42.334187 4564 ts_tablet_manager.cc:1939] T 8459b86c68bc4c07bbde48afb822961b P 65a8c0f690454f449ae088af9b75bb98: Deleting consensus metadata
I20260808 10:07:42.334512 3944 catalog_manager.cc:4841] TS 65a8c0f690454f449ae088af9b75bb98 (127.2.30.131:33363): tablet 8459b86c68bc4c07bbde48afb822961b (table test-table [id=77f123dfa86547558e2585d7ce47c069]) successfully deleted
I20260808 10:07:42.335181 4563 ts_tablet_manager.cc:1918] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:07:42.335255 4563 log.cc:1196] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-0/wal/wals/8459b86c68bc4c07bbde48afb822961b
I20260808 10:07:42.335295 4566 ts_tablet_manager.cc:1918] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:07:42.335366 4566 log.cc:1196] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/8459b86c68bc4c07bbde48afb822961b
I20260808 10:07:42.335495 4563 ts_tablet_manager.cc:1939] T 8459b86c68bc4c07bbde48afb822961b P b693b00a8aee4f58b0248ad4b8a66021: Deleting consensus metadata
I20260808 10:07:42.335507 4099 tablet_service.cc:1467] Processing CreateTablet for tablet 97aeca14f028477cb740b4dc76caab54 (DEFAULT_TABLE table=test-table [id=10ef7195e9f3422faf1ec3eef263d759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:42.335640 4566 ts_tablet_manager.cc:1939] T 8459b86c68bc4c07bbde48afb822961b P 203debce33054051af9850803ba3fce0: Deleting consensus metadata
I20260808 10:07:42.335676 4099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97aeca14f028477cb740b4dc76caab54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:42.335879 3945 catalog_manager.cc:4841] TS b693b00a8aee4f58b0248ad4b8a66021 (127.2.30.129:32903): tablet 8459b86c68bc4c07bbde48afb822961b (table test-table [id=77f123dfa86547558e2585d7ce47c069]) successfully deleted
I20260808 10:07:42.336755 4575 tablet_bootstrap.cc:492] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021: Bootstrap starting.
I20260808 10:07:42.336964 3945 catalog_manager.cc:4841] TS 203debce33054051af9850803ba3fce0 (127.2.30.130:43009): tablet 8459b86c68bc4c07bbde48afb822961b (table test-table [id=77f123dfa86547558e2585d7ce47c069]) successfully deleted
I20260808 10:07:42.337090 4575 tablet_bootstrap.cc:654] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:42.337092 4243 tablet_service.cc:1467] Processing CreateTablet for tablet 97aeca14f028477cb740b4dc76caab54 (DEFAULT_TABLE table=test-table [id=10ef7195e9f3422faf1ec3eef263d759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:42.337196 4243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97aeca14f028477cb740b4dc76caab54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:42.337463 4387 tablet_service.cc:1467] Processing CreateTablet for tablet 97aeca14f028477cb740b4dc76caab54 (DEFAULT_TABLE table=test-table [id=10ef7195e9f3422faf1ec3eef263d759]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:42.337565 4387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97aeca14f028477cb740b4dc76caab54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:42.337867 4575 tablet_bootstrap.cc:492] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021: No bootstrap required, opened a new log
I20260808 10:07:42.337906 4575 ts_tablet_manager.cc:1397] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:42.338001 4575 raft_consensus.cc:348] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.338056 4575 raft_consensus.cc:374] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:42.338071 4575 raft_consensus.cc:729] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b693b00a8aee4f58b0248ad4b8a66021, State: Initialized, Role: FOLLOWER
I20260808 10:07:42.338142 4575 consensus_queue.cc:260] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.338270 4575 ts_tablet_manager.cc:1428] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:42.338490 4576 tablet_bootstrap.cc:492] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0: Bootstrap starting.
I20260808 10:07:42.338975 4580 tablet_bootstrap.cc:492] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98: Bootstrap starting.
I20260808 10:07:42.339180 4576 tablet_bootstrap.cc:654] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:42.339411 4580 tablet_bootstrap.cc:654] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:42.339919 4576 tablet_bootstrap.cc:492] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0: No bootstrap required, opened a new log
I20260808 10:07:42.339968 4576 ts_tablet_manager.cc:1397] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:42.340071 4580 tablet_bootstrap.cc:492] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98: No bootstrap required, opened a new log
I20260808 10:07:42.340091 4576 raft_consensus.cc:348] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.340143 4576 raft_consensus.cc:374] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:42.340162 4576 raft_consensus.cc:729] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 203debce33054051af9850803ba3fce0, State: Initialized, Role: FOLLOWER
I20260808 10:07:42.340148 4580 ts_tablet_manager.cc:1397] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 10:07:42.340224 4576 consensus_queue.cc:260] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.340286 4580 raft_consensus.cc:348] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.340370 4580 raft_consensus.cc:374] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:42.340364 4576 ts_tablet_manager.cc:1428] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:42.340399 4580 raft_consensus.cc:729] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65a8c0f690454f449ae088af9b75bb98, State: Initialized, Role: FOLLOWER
I20260808 10:07:42.340456 4580 consensus_queue.cc:260] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.340556 4580 ts_tablet_manager.cc:1428] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:42.625916 4543 raft_consensus.cc:482] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:07:42.626065 4543 raft_consensus.cc:504] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.626322 4543 leader_election.cc:288] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b693b00a8aee4f58b0248ad4b8a66021 (127.2.30.129:32903), 203debce33054051af9850803ba3fce0 (127.2.30.130:43009)
I20260808 10:07:42.626622 4264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97aeca14f028477cb740b4dc76caab54" candidate_uuid: "65a8c0f690454f449ae088af9b75bb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "203debce33054051af9850803ba3fce0" is_pre_election: true
I20260808 10:07:42.626646 4119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97aeca14f028477cb740b4dc76caab54" candidate_uuid: "65a8c0f690454f449ae088af9b75bb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b693b00a8aee4f58b0248ad4b8a66021" is_pre_election: true
I20260808 10:07:42.626752 4119 raft_consensus.cc:2454] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65a8c0f690454f449ae088af9b75bb98 in term 0.
I20260808 10:07:42.626752 4264 raft_consensus.cc:2454] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65a8c0f690454f449ae088af9b75bb98 in term 0.
I20260808 10:07:42.626940 4340 leader_election.cc:302] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65a8c0f690454f449ae088af9b75bb98, b693b00a8aee4f58b0248ad4b8a66021; no voters:
I20260808 10:07:42.627095 4543 raft_consensus.cc:2781] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:07:42.627151 4543 raft_consensus.cc:482] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:07:42.627171 4543 raft_consensus.cc:3037] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:42.657940 4543 raft_consensus.cc:504] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.658191 4543 leader_election.cc:288] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [CANDIDATE]: Term 1 election: Requested vote from peers b693b00a8aee4f58b0248ad4b8a66021 (127.2.30.129:32903), 203debce33054051af9850803ba3fce0 (127.2.30.130:43009)
I20260808 10:07:42.658584 4264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97aeca14f028477cb740b4dc76caab54" candidate_uuid: "65a8c0f690454f449ae088af9b75bb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "203debce33054051af9850803ba3fce0"
I20260808 10:07:42.658560 4119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97aeca14f028477cb740b4dc76caab54" candidate_uuid: "65a8c0f690454f449ae088af9b75bb98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b693b00a8aee4f58b0248ad4b8a66021"
I20260808 10:07:42.658682 4119 raft_consensus.cc:3037] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:42.658690 4264 raft_consensus.cc:3037] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:42.659626 4119 raft_consensus.cc:2454] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65a8c0f690454f449ae088af9b75bb98 in term 1.
I20260808 10:07:42.659643 4264 raft_consensus.cc:2454] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65a8c0f690454f449ae088af9b75bb98 in term 1.
I20260808 10:07:42.659849 4340 leader_election.cc:302] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65a8c0f690454f449ae088af9b75bb98, b693b00a8aee4f58b0248ad4b8a66021; no voters:
I20260808 10:07:42.660051 4543 raft_consensus.cc:2781] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:42.660290 4543 raft_consensus.cc:686] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [term 1 LEADER]: Becoming Leader. State: Replica: 65a8c0f690454f449ae088af9b75bb98, State: Running, Role: LEADER
I20260808 10:07:42.660365 4543 consensus_queue.cc:237] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } }
I20260808 10:07:42.661060 3958 catalog_manager.cc:5495] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 65a8c0f690454f449ae088af9b75bb98 (127.2.30.131). New cstate: current_term: 1 leader_uuid: "65a8c0f690454f449ae088af9b75bb98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65a8c0f690454f449ae088af9b75bb98" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33363 } health_report { overall_health: HEALTHY } } }
I20260808 10:07:42.697072 4592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:42.694636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56596 (local address 127.2.30.131:33363)
0808 10:07:42.694832 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:42.694834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:42.694841 (+ 7us) server_negotiation.cc:407] Connection header received
0808 10:07:42.694861 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:42.694862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:42.694872 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:42.694931 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:42.695121 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:42.695626 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:42.696337 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:42.696588 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:42.696654 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:42.696820 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:42.696848 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:42.696885 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:42.696919 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 10:07:42.696937 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":129,"threads_started":1}
I20260808 10:07:42.702888 4592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:42.700169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:48889 (local address 127.2.30.131:33363)
0808 10:07:42.700222 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:42.700225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:42.700483 (+ 258us) server_negotiation.cc:407] Connection header received
0808 10:07:42.700512 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:42.700513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:42.700526 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:42.700583 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:42.700866 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:42.701403 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:42.702226 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:42.702458 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:42.702529 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:42.702667 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:42.702693 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:42.702734 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:42.702794 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 10:07:42.702815 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 10:07:42.702906 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:42.700055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:33363 (local address 127.2.30.131:48889)
0808 10:07:42.700326 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:42.700345 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:42.700435 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:42.700605 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:42.700606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:42.700608 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:07:42.700696 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:42.700698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:42.701518 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:42.701519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:42.702079 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:42.702081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:42.702538 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:42.702539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:42.702592 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:42.702596 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:07:42.702748 (+ 152us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:07:42.702750 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:07:42.702791 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 10:07:42.702832 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":123,"threads_started":1}
I20260808 10:07:42.705740 4119 raft_consensus.cc:1261] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021 [term 1 FOLLOWER]: Refusing update from remote peer 65a8c0f690454f449ae088af9b75bb98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:42.705922 4264 raft_consensus.cc:1261] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Refusing update from remote peer 65a8c0f690454f449ae088af9b75bb98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:42.706221 4549 consensus_queue.cc:1035] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "203debce33054051af9850803ba3fce0" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 43009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:42.706442 4503 consensus_queue.cc:1035] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b693b00a8aee4f58b0248ad4b8a66021" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:07:42.711450 4540 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 10:07:44.238201 3958 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:07:44.238313 3958 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:07:44.239477 3958 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9bf3086c1ef64675bfdc48fe63433d6b: Sending DeleteTablet for 3 replicas of tablet 97aeca14f028477cb740b4dc76caab54
I20260808 10:07:44.239754 4099 tablet_service.cc:1514] Processing DeleteTablet for tablet 97aeca14f028477cb740b4dc76caab54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:44 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:60216
I20260808 10:07:44.239796 4243 tablet_service.cc:1514] Processing DeleteTablet for tablet 97aeca14f028477cb740b4dc76caab54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:44 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:34768
I20260808 10:07:44.239864 4386 tablet_service.cc:1514] Processing DeleteTablet for tablet 97aeca14f028477cb740b4dc76caab54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:44 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:56570
I20260808 10:07:44.240016 4619 tablet_replica.cc:330] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0: stopping tablet replica
I20260808 10:07:44.240070 4619 raft_consensus.cc:2229] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:07:44.240108 4619 raft_consensus.cc:2258] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:44.240163 4618 tablet_replica.cc:330] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021: stopping tablet replica
I20260808 10:07:44.240204 4620 tablet_replica.cc:330] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98: stopping tablet replica
I20260808 10:07:44.240239 4618 raft_consensus.cc:2229] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:07:44.240254 4620 raft_consensus.cc:2229] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:07:44.240285 4618 raft_consensus.cc:2258] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:44.240545 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 4032
I20260808 10:07:44.240557 4618 ts_tablet_manager.cc:1905] T 97aeca14f028477cb740b4dc76caab54 P b693b00a8aee4f58b0248ad4b8a66021: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:44.240479 4620 raft_consensus.cc:2258] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:44.241940 4620 ts_tablet_manager.cc:1905] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:44.243827 4620 ts_tablet_manager.cc:1918] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:07:44.243882 4620 log.cc:1196] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/97aeca14f028477cb740b4dc76caab54
I20260808 10:07:44.244124 4620 ts_tablet_manager.cc:1939] T 97aeca14f028477cb740b4dc76caab54 P 65a8c0f690454f449ae088af9b75bb98: Deleting consensus metadata
I20260808 10:07:44.244802 3944 catalog_manager.cc:4841] TS 65a8c0f690454f449ae088af9b75bb98 (127.2.30.131:33363): tablet 97aeca14f028477cb740b4dc76caab54 (table test-table [id=10ef7195e9f3422faf1ec3eef263d759]) successfully deleted
I20260808 10:07:44.247259 4619 ts_tablet_manager.cc:1905] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:44.248791 4619 ts_tablet_manager.cc:1918] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 10:07:44.248854 4619 log.cc:1196] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/97aeca14f028477cb740b4dc76caab54
I20260808 10:07:44.249114 4619 ts_tablet_manager.cc:1939] T 97aeca14f028477cb740b4dc76caab54 P 203debce33054051af9850803ba3fce0: Deleting consensus metadata
I20260808 10:07:44.249505 3945 catalog_manager.cc:4841] TS 203debce33054051af9850803ba3fce0 (127.2.30.130:43009): tablet 97aeca14f028477cb740b4dc76caab54 (table test-table [id=10ef7195e9f3422faf1ec3eef263d759]) successfully deleted
W20260808 10:07:44.250454 3945 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:32903 (error 108)
I20260808 10:07:44.250762 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 4175
I20260808 10:07:44.253393 4622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:44.250924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:32903 (local address 127.0.0.1:60254)
0808 10:07:44.253269 (+ 2345us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:44.253336 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:32903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2311,"thread_start_us":85,"threads_started":1}
W20260808 10:07:44.253561 3945 catalog_manager.cc:4584] TS b693b00a8aee4f58b0248ad4b8a66021 (127.2.30.129:32903): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 97aeca14f028477cb740b4dc76caab54: Network error: Client connection negotiation failed: client connection to 127.2.30.129:32903: connect: Connection refused (error 111)
I20260808 10:07:44.257211 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 4320
I20260808 10:07:44.264967 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 3926
2026-08-08T10:07:44Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5929 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:44 dist-test-slave-w206 krb5kdc[4625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:44 dist-test-slave-w206 krb5kdc[4625](info): set up 2 sockets
Aug 08 10:07:44 dist-test-slave-w206 krb5kdc[4625](info): commencing operation
krb5kdc: starting...
W20260808 10:07:45.343524 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:07:45 dist-test-slave-w206 krb5kdc[4625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:07:45.477808 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:41139
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:41139
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:45.565655 4638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:45.565896 4638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:45.565922 4638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:45.567097 4638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:45.567140 4638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:45.567157 4638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:45.567173 4638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:45.567195 4638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:45.568688 4638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:41139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:41139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:45.568903 4638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:45.569092 4638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:07:45.574905 4638 server_base.cc:1034] running on GCE node
W20260808 10:07:45.574883 4644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:45.574872 4645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:45.574883 4648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:45.575742 4638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:45.576093 4638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:45.577301 4638 hybrid_clock.cc:648] HybridClock initialized: now 1786183665577293 us; error 54 us; skew 500 ppm
Aug 08 10:07:45 dist-test-slave-w206 krb5kdc[4625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183665, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:45.580245 4638 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:07:45.580741 4638 webserver.cc:460] Webserver started at http://127.2.30.190:34195/ using document root <none> and password file <none>
I20260808 10:07:45.581030 4638 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:45.581104 4638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:45.581254 4638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:45.582347 4638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "870e8920f7b2477fbd8b1b27dafc3e7e"
format_stamp: "Formatted at 2026-08-08 10:07:45 on dist-test-slave-w206"
server_key: "099f70433dfa0a2784aa340078806016"
server_key_iv: "f8705628026a05b18fdd84c2cccf81ac"
server_key_version: "encryptionkey@0"
I20260808 10:07:45.582706 4638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "870e8920f7b2477fbd8b1b27dafc3e7e"
format_stamp: "Formatted at 2026-08-08 10:07:45 on dist-test-slave-w206"
server_key: "099f70433dfa0a2784aa340078806016"
server_key_iv: "f8705628026a05b18fdd84c2cccf81ac"
server_key_version: "encryptionkey@0"
I20260808 10:07:45.584213 4638 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:07:45.585091 4655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:45.585325 4638 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:45.585428 4638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "870e8920f7b2477fbd8b1b27dafc3e7e"
format_stamp: "Formatted at 2026-08-08 10:07:45 on dist-test-slave-w206"
server_key: "099f70433dfa0a2784aa340078806016"
server_key_iv: "f8705628026a05b18fdd84c2cccf81ac"
server_key_version: "encryptionkey@0"
I20260808 10:07:45.585512 4638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:45.601547 4638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:45.603691 4638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:45.603868 4638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:45.608132 4638 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:41139
I20260808 10:07:45.608180 4709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:41139 every 8 connection(s)
I20260808 10:07:45.608560 4638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:45.609498 4710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:45.612237 4710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e: Bootstrap starting.
I20260808 10:07:45.612819 4710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:45.613068 4710 log.cc:824] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:45.613828 4710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e: No bootstrap required, opened a new log
I20260808 10:07:45.613987 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 4638
I20260808 10:07:45.614076 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:07:45.614261 2170 external_mini_cluster.cc:1428] Setting key 23b55a6917d0200dae801e2a52aa4a3c
I20260808 10:07:45.615998 4710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "870e8920f7b2477fbd8b1b27dafc3e7e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41139 } }
I20260808 10:07:45.616190 4710 raft_consensus.cc:374] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:45.616231 4710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 870e8920f7b2477fbd8b1b27dafc3e7e, State: Initialized, Role: FOLLOWER
I20260808 10:07:45.616410 4710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "870e8920f7b2477fbd8b1b27dafc3e7e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41139 } }
I20260808 10:07:45.616510 4710 raft_consensus.cc:388] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:45.616549 4710 raft_consensus.cc:482] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:45.616593 4710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:45.617637 4710 raft_consensus.cc:504] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "870e8920f7b2477fbd8b1b27dafc3e7e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41139 } }
I20260808 10:07:45.617731 4710 leader_election.cc:302] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 870e8920f7b2477fbd8b1b27dafc3e7e; no voters:
I20260808 10:07:45.617919 4710 leader_election.cc:288] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:45.617987 4715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:45.618186 4715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [term 1 LEADER]: Becoming Leader. State: Replica: 870e8920f7b2477fbd8b1b27dafc3e7e, State: Running, Role: LEADER
I20260808 10:07:45.618384 4715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "870e8920f7b2477fbd8b1b27dafc3e7e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41139 } }
I20260808 10:07:45.618813 4716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "870e8920f7b2477fbd8b1b27dafc3e7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "870e8920f7b2477fbd8b1b27dafc3e7e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41139 } } }
I20260808 10:07:45.618811 4717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 870e8920f7b2477fbd8b1b27dafc3e7e. Latest consensus state: current_term: 1 leader_uuid: "870e8920f7b2477fbd8b1b27dafc3e7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "870e8920f7b2477fbd8b1b27dafc3e7e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41139 } } }
I20260808 10:07:45.618927 4716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:45.618937 4717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:45.618209 4710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:45.619498 4720 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:45.620002 4720 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:45.621562 4720 catalog_manager.cc:1347] Generated new cluster ID: a7df645da9c0470cb073ace48f7e256b
I20260808 10:07:45.621609 4720 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:07:45 dist-test-slave-w206 krb5kdc[4625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:45.626406 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:45.614860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57532 (local address 127.2.30.190:41139)
0808 10:07:45.615069 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:45.615077 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:45.615108 (+ 31us) server_negotiation.cc:407] Connection header received
0808 10:07:45.615983 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:45.615991 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:45.616041 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:45.616158 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:45.617006 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:45.618235 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:45.619360 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:45.620058 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:45.621287 (+ 1229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:45.622964 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:45.622968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:45.622969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:45.623000 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:45.624773 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:45.625158 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:45.625169 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:45.625169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:45.625278 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:45.625505 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:45.625505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:45.625506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:45.625783 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:45.625969 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:45.626039 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 10:07:45.626206 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260808 10:07:45.644613 4720 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:45.645219 4720 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:45.653373 4720 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 870e8920f7b2477fbd8b1b27dafc3e7e: Generated new TSK 0
I20260808 10:07:45.653616 4720 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:07:45.688344 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41139
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:45.780412 4736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:45.780640 4736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:45.780666 4736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:45.781966 4736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:45.782013 4736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:45.782042 4736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:07:45.783679 4736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:45.783965 4736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:45.784229 4736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:45.790032 4744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:45.790016 4742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:45.790016 4741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:45.790328 4736 server_base.cc:1034] running on GCE node
I20260808 10:07:45.790740 4736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:45.791132 4736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:45.792310 4736 hybrid_clock.cc:648] HybridClock initialized: now 1786183665792318 us; error 73 us; skew 500 ppm
Aug 08 10:07:45 dist-test-slave-w206 krb5kdc[4625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183665, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:45.795259 4736 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:07:45.795717 4736 webserver.cc:460] Webserver started at http://127.2.30.129:40789/ using document root <none> and password file <none>
I20260808 10:07:45.796007 4736 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:45.796090 4736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:45.796244 4736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:45.797386 4736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "840247e8e8914aa1a2d1830d11a3999a"
format_stamp: "Formatted at 2026-08-08 10:07:45 on dist-test-slave-w206"
server_key: "40d643b3613a865ae64ccc32983ae11f"
server_key_iv: "d3b0d7e3809a32b3bf656a9f1a4d8f84"
server_key_version: "encryptionkey@0"
I20260808 10:07:45.797763 4736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "840247e8e8914aa1a2d1830d11a3999a"
format_stamp: "Formatted at 2026-08-08 10:07:45 on dist-test-slave-w206"
server_key: "40d643b3613a865ae64ccc32983ae11f"
server_key_iv: "d3b0d7e3809a32b3bf656a9f1a4d8f84"
server_key_version: "encryptionkey@0"
I20260808 10:07:45.799372 4736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:07:45.800168 4751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:45.800392 4736 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:45.800504 4736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "840247e8e8914aa1a2d1830d11a3999a"
format_stamp: "Formatted at 2026-08-08 10:07:45 on dist-test-slave-w206"
server_key: "40d643b3613a865ae64ccc32983ae11f"
server_key_iv: "d3b0d7e3809a32b3bf656a9f1a4d8f84"
server_key_version: "encryptionkey@0"
I20260808 10:07:45.800582 4736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:45.824792 4736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:45.826891 4736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:45.827082 4736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:45.827361 4736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:45.827688 4736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:45.827750 4736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:45.827793 4736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:45.827834 4736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:45.833607 4736 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:35911
I20260808 10:07:45.833686 4868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:35911 every 8 connection(s)
I20260808 10:07:45.834026 4736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:07:45.835498 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 4736
I20260808 10:07:45.835602 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:07:45.835765 2170 external_mini_cluster.cc:1428] Setting key 6afc69994b10ac70cc66e618b210cb35
Aug 08 10:07:45 dist-test-slave-w206 krb5kdc[4625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183665, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
I20260808 10:07:45.845247 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:45.834893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:50729 (local address 127.2.30.190:41139)
0808 10:07:45.834962 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:45.834965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:45.835181 (+ 216us) server_negotiation.cc:407] Connection header received
0808 10:07:45.835911 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:45.835913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:45.835928 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:45.836006 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:45.837135 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:45.837901 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:45.838945 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:45.839195 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:45.839288 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:45.841905 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:45.841915 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:45.841916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:45.841926 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:45.843617 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:45.844142 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:45.844143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:45.844144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:45.844218 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:45.844501 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:45.844502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:45.844503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:45.844637 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:45.844727 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:45.845026 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 10:07:45.845128 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:07:45.845468 4871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:45.834865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41139 (local address 127.2.30.129:50729)
0808 10:07:45.835046 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:45.835091 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:45.835785 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:45.836101 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:45.836106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:45.836834 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:45.837030 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:45.837035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:45.837997 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:45.837997 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:45.838848 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:45.838850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:45.839303 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:45.839304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:45.839463 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:45.840018 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:45.840029 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:45.841798 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:45.843784 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:45.843793 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:45.843796 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:45.844035 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:45.844309 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:45.844310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:45.844310 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:45.844427 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:45.844733 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:45.844746 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:45.844887 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 10:07:45.845006 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 10:07:45.845207 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260808 10:07:45.846053 4869 heartbeater.cc:347] Connected to a master server at 127.2.30.190:41139
I20260808 10:07:45.846154 4869 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:45.846385 4869 heartbeater.cc:511] Master 127.2.30.190:41139 requested a full tablet report, sending...
I20260808 10:07:45.847011 4673 ts_manager.cc:194] Registered new tserver with Master: 840247e8e8914aa1a2d1830d11a3999a (127.2.30.129:35911)
I20260808 10:07:45.847677 4673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:50729
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:07:45.864176 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41139
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:45.947956 4874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:45.948168 4874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:45.948194 4874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:45.949278 4874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:45.949313 4874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:45.949347 4874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:07:45.950747 4874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:45.950940 4874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:45.951210 4874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:45.956274 4879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:45.956349 4880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:45.956506 4874 server_base.cc:1034] running on GCE node
W20260808 10:07:45.956274 4882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:45.956754 4874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:45.957013 4874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:45.958165 4874 hybrid_clock.cc:648] HybridClock initialized: now 1786183665958146 us; error 42 us; skew 500 ppm
Aug 08 10:07:45 dist-test-slave-w206 krb5kdc[4625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183665, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:45.960952 4874 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:07:45.961434 4874 webserver.cc:460] Webserver started at http://127.2.30.130:45107/ using document root <none> and password file <none>
I20260808 10:07:45.961688 4874 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:45.961764 4874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:45.961920 4874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:45.963055 4874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "a55a2de9af454bc8bd10e767a8f7e790"
format_stamp: "Formatted at 2026-08-08 10:07:45 on dist-test-slave-w206"
server_key: "9aea0c0feb0a4753005ee6abac63ff47"
server_key_iv: "2ba666d4dab97e84b08e9c5c3bb30834"
server_key_version: "encryptionkey@0"
I20260808 10:07:45.963428 4874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "a55a2de9af454bc8bd10e767a8f7e790"
format_stamp: "Formatted at 2026-08-08 10:07:45 on dist-test-slave-w206"
server_key: "9aea0c0feb0a4753005ee6abac63ff47"
server_key_iv: "2ba666d4dab97e84b08e9c5c3bb30834"
server_key_version: "encryptionkey@0"
I20260808 10:07:45.964838 4874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 10:07:45.965669 4889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:45.965916 4874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:07:45.966017 4874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "a55a2de9af454bc8bd10e767a8f7e790"
format_stamp: "Formatted at 2026-08-08 10:07:45 on dist-test-slave-w206"
server_key: "9aea0c0feb0a4753005ee6abac63ff47"
server_key_iv: "2ba666d4dab97e84b08e9c5c3bb30834"
server_key_version: "encryptionkey@0"
I20260808 10:07:45.966109 4874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:46.004191 4874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:46.006577 4874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:46.006755 4874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:46.007038 4874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:46.007404 4874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:46.007467 4874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:46.007509 4874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:46.007566 4874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:46.013623 4874 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:40955
I20260808 10:07:46.013702 5006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:40955 every 8 connection(s)
I20260808 10:07:46.014063 4874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:07:46.020419 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 4874
I20260808 10:07:46.020505 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:07:46.020720 2170 external_mini_cluster.cc:1428] Setting key b0c02625c1206d792a74cc818649d56d
Aug 08 10:07:46 dist-test-slave-w206 krb5kdc[4625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183665, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:46.025727 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:46.014883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:41811 (local address 127.2.30.190:41139)
0808 10:07:46.014958 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:46.014961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:46.015228 (+ 267us) server_negotiation.cc:407] Connection header received
0808 10:07:46.015802 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:46.015803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:46.015817 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:46.015889 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:46.016935 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:46.017489 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:46.018558 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:46.018761 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:46.018846 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:46.021973 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:46.021975 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:46.021975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:46.021984 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:46.023884 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:46.024588 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:46.024589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:46.024589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:46.024647 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:46.024974 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:46.024975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:46.024975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:46.025094 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:46.025146 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:46.025574 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 10:07:46.025657 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:07:46.026087 5009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:46.014853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41139 (local address 127.2.30.130:41811)
0808 10:07:46.015096 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:46.015135 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:46.015640 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:46.015979 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:46.015983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:46.016620 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:46.016828 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:46.016833 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:46.017606 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:46.017607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:46.018466 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:46.018469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:46.018903 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:46.018905 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:46.019136 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:46.019836 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:46.019844 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:46.021868 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:46.024001 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:46.024012 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:46.024015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:46.024501 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:46.024721 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:46.024722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:46.024722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:46.024898 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:46.025173 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:46.025180 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:46.025416 (+ 236us) client_negotiation.cc:769] Sending connection context
0808 10:07:46.025553 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 10:07:46.025766 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":130,"threads_started":1}
I20260808 10:07:46.026692 5007 heartbeater.cc:347] Connected to a master server at 127.2.30.190:41139
I20260808 10:07:46.026782 5007 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:46.027011 5007 heartbeater.cc:511] Master 127.2.30.190:41139 requested a full tablet report, sending...
I20260808 10:07:46.027484 4673 ts_manager.cc:194] Registered new tserver with Master: a55a2de9af454bc8bd10e767a8f7e790 (127.2.30.130:40955)
I20260808 10:07:46.027878 4673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:41811
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:07:46.051168 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41139
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:46.140594 5012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:46.140836 5012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:46.140863 5012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:46.142035 5012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:46.142081 5012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:46.142112 5012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:07:46.143731 5012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:46.143973 5012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:46.144268 5012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:46.149902 5017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:46.149947 5020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:46.150027 5018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:46.150281 5012 server_base.cc:1034] running on GCE node
I20260808 10:07:46.150691 5012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:46.151054 5012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:46.152406 5012 hybrid_clock.cc:648] HybridClock initialized: now 1786183666152376 us; error 61 us; skew 500 ppm
Aug 08 10:07:46 dist-test-slave-w206 krb5kdc[4625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183666, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:46.155524 5012 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:07:46.156073 5012 webserver.cc:460] Webserver started at http://127.2.30.131:33347/ using document root <none> and password file <none>
I20260808 10:07:46.156387 5012 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:46.156466 5012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:46.156629 5012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:46.157796 5012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "8a681aa8f2d44375aa97b77a9739d647"
format_stamp: "Formatted at 2026-08-08 10:07:46 on dist-test-slave-w206"
server_key: "c21ac9b7d899ba613e50c3540e1576ce"
server_key_iv: "fd2266d35992f828c0089c590e5040ce"
server_key_version: "encryptionkey@0"
I20260808 10:07:46.158164 5012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "8a681aa8f2d44375aa97b77a9739d647"
format_stamp: "Formatted at 2026-08-08 10:07:46 on dist-test-slave-w206"
server_key: "c21ac9b7d899ba613e50c3540e1576ce"
server_key_iv: "fd2266d35992f828c0089c590e5040ce"
server_key_version: "encryptionkey@0"
I20260808 10:07:46.159719 5012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:07:46.160632 5027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:46.160815 5012 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:46.160939 5012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "8a681aa8f2d44375aa97b77a9739d647"
format_stamp: "Formatted at 2026-08-08 10:07:46 on dist-test-slave-w206"
server_key: "c21ac9b7d899ba613e50c3540e1576ce"
server_key_iv: "fd2266d35992f828c0089c590e5040ce"
server_key_version: "encryptionkey@0"
I20260808 10:07:46.161026 5012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:46.171803 5012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:46.174281 5012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:46.174525 5012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:46.174805 5012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:46.175230 5012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:46.175293 5012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:46.175400 5012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:46.175432 5012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:46.182317 5012 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:39781
I20260808 10:07:46.182368 5147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:39781 every 8 connection(s)
I20260808 10:07:46.182727 5012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:07:46.187592 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 5012
I20260808 10:07:46.187774 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:07:46.188011 2170 external_mini_cluster.cc:1428] Setting key e830e39df2b3904b147ae97e243f5ce4
Aug 08 10:07:46 dist-test-slave-w206 krb5kdc[4625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183666, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:46.193514 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:46.183602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:55225 (local address 127.2.30.190:41139)
0808 10:07:46.183683 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:46.183685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:46.183916 (+ 231us) server_negotiation.cc:407] Connection header received
0808 10:07:46.184457 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:46.184459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:46.184472 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:46.184538 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:46.185555 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:46.186131 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:46.187289 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:46.187588 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:46.187677 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:46.190451 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:46.190452 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:46.190453 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:46.190460 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:46.192029 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:46.192501 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:46.192502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:46.192503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:46.192564 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:46.192827 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:46.192829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:46.192829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:46.192948 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:46.193015 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:46.193364 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 10:07:46.193445 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 10:07:46.193717 5150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:46.183553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41139 (local address 127.2.30.131:55225)
0808 10:07:46.183747 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:46.183795 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:46.184339 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:46.184597 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:46.184601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:46.185231 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:46.185454 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:46.185459 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:46.186237 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:46.186237 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:46.187203 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:46.187206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:46.187736 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:46.187737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:46.187909 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:46.188537 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:46.188555 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:46.190359 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:46.192112 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:46.192118 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:46.192121 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:46.192381 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:46.192640 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:46.192641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:46.192641 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:46.192757 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:46.193055 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:46.193062 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:46.193196 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 10:07:46.193345 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 10:07:46.193493 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
I20260808 10:07:46.194353 5148 heartbeater.cc:347] Connected to a master server at 127.2.30.190:41139
I20260808 10:07:46.194440 5148 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:46.194648 5148 heartbeater.cc:511] Master 127.2.30.190:41139 requested a full tablet report, sending...
I20260808 10:07:46.195099 4673 ts_manager.cc:194] Registered new tserver with Master: 8a681aa8f2d44375aa97b77a9739d647 (127.2.30.131:39781)
I20260808 10:07:46.195447 4673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:55225
I20260808 10:07:46.200515 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 10:07:46.214454 4714 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:46.213227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51122 (local address 127.2.30.190:41139)
0808 10:07:46.213360 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:46.213363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:46.213480 (+ 117us) server_negotiation.cc:407] Connection header received
0808 10:07:46.213582 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:46.213584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:46.213596 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:46.213672 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:46.214390 (+ 718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51122: BlockingRecv error: recv got EOF from 127.0.0.1:51122 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 10:07:46.215626 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 4736
I20260808 10:07:46.221706 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 4874
I20260808 10:07:46.229349 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 5012
I20260808 10:07:46.236375 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 4638
2026-08-08T10:07:46Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1969 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:46 dist-test-slave-w206 krb5kdc[5161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:46 dist-test-slave-w206 krb5kdc[5161](info): set up 2 sockets
Aug 08 10:07:46 dist-test-slave-w206 krb5kdc[5161](info): commencing operation
krb5kdc: starting...
W20260808 10:07:47.308109 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:07:47 dist-test-slave-w206 krb5kdc[5161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:07:47.442605 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:44257
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:45173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:44257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:47.533300 5175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:47.533583 5175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:47.533627 5175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:47.534915 5175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:47.535032 5175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:47.535072 5175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:47.535104 5175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:47.535133 5175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:47.536834 5175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:45173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:44257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:44257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:47.537137 5175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:47.537386 5175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:47.542747 5180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:47.542716 5183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:47.542737 5181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:47.543051 5175 server_base.cc:1034] running on GCE node
I20260808 10:07:47.543314 5175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:47.543612 5175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:47.544790 5175 hybrid_clock.cc:648] HybridClock initialized: now 1786183667544770 us; error 31 us; skew 500 ppm
Aug 08 10:07:47 dist-test-slave-w206 krb5kdc[5161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183667, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:47.552744 5175 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:07:47.553345 5175 webserver.cc:460] Webserver started at http://127.2.30.190:44701/ using document root <none> and password file <none>
I20260808 10:07:47.553615 5175 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:47.553690 5175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:47.553855 5175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:47.555136 5175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "74ab44c6d2f64e3ba7056e68360623ac"
format_stamp: "Formatted at 2026-08-08 10:07:47 on dist-test-slave-w206"
server_key: "9f52237f9edcf3065f4382ef5f651a0f"
server_key_iv: "dbd05b5aa357e6e3cde5e07c3bb13f10"
server_key_version: "encryptionkey@0"
I20260808 10:07:47.555532 5175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "74ab44c6d2f64e3ba7056e68360623ac"
format_stamp: "Formatted at 2026-08-08 10:07:47 on dist-test-slave-w206"
server_key: "9f52237f9edcf3065f4382ef5f651a0f"
server_key_iv: "dbd05b5aa357e6e3cde5e07c3bb13f10"
server_key_version: "encryptionkey@0"
I20260808 10:07:47.557055 5175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:47.557967 5190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:47.558166 5175 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:47.558320 5175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "74ab44c6d2f64e3ba7056e68360623ac"
format_stamp: "Formatted at 2026-08-08 10:07:47 on dist-test-slave-w206"
server_key: "9f52237f9edcf3065f4382ef5f651a0f"
server_key_iv: "dbd05b5aa357e6e3cde5e07c3bb13f10"
server_key_version: "encryptionkey@0"
I20260808 10:07:47.558413 5175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:47.577780 5175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:47.580415 5175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:47.580590 5175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:47.585078 5175 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:44257
I20260808 10:07:47.585117 5243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:44257 every 8 connection(s)
I20260808 10:07:47.585511 5175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:47.586457 5244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:47.588930 5244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac: Bootstrap starting.
I20260808 10:07:47.589388 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 5175
I20260808 10:07:47.589484 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:07:47.589673 2170 external_mini_cluster.cc:1428] Setting key b5780955b4f6d92c7569a8c5754f3025
I20260808 10:07:47.589679 5244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:47.590049 5244 log.cc:824] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:47.590895 5244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac: No bootstrap required, opened a new log
I20260808 10:07:47.593328 5244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ab44c6d2f64e3ba7056e68360623ac" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 44257 } }
I20260808 10:07:47.593513 5244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:47.593549 5244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74ab44c6d2f64e3ba7056e68360623ac, State: Initialized, Role: FOLLOWER
I20260808 10:07:47.593676 5244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ab44c6d2f64e3ba7056e68360623ac" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 44257 } }
I20260808 10:07:47.593758 5244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:47.593788 5244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:47.593832 5244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:47.594524 5244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ab44c6d2f64e3ba7056e68360623ac" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 44257 } }
I20260808 10:07:47.594666 5244 leader_election.cc:302] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74ab44c6d2f64e3ba7056e68360623ac; no voters:
I20260808 10:07:47.594880 5244 leader_election.cc:288] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:47.594978 5249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:47.595183 5249 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [term 1 LEADER]: Becoming Leader. State: Replica: 74ab44c6d2f64e3ba7056e68360623ac, State: Running, Role: LEADER
I20260808 10:07:47.595216 5244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:47.595363 5249 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ab44c6d2f64e3ba7056e68360623ac" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 44257 } }
I20260808 10:07:47.595729 5250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74ab44c6d2f64e3ba7056e68360623ac. Latest consensus state: current_term: 1 leader_uuid: "74ab44c6d2f64e3ba7056e68360623ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ab44c6d2f64e3ba7056e68360623ac" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 44257 } } }
I20260808 10:07:47.595826 5250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:47.596055 5254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:47.596596 5251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74ab44c6d2f64e3ba7056e68360623ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74ab44c6d2f64e3ba7056e68360623ac" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 44257 } } }
I20260808 10:07:47.596778 5251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac [sys.catalog]: This master's current role is: LEADER
Aug 08 10:07:47 dist-test-slave-w206 krb5kdc[5161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:47.597352 5254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:47.599933 5254 catalog_manager.cc:1347] Generated new cluster ID: 0bf6da64fd254d1b8309f5279d8f17f8
I20260808 10:07:47.599993 5254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:47.601577 5248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:47.590236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53484 (local address 127.2.30.190:44257)
0808 10:07:47.590480 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:47.590490 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:47.590532 (+ 42us) server_negotiation.cc:407] Connection header received
0808 10:07:47.591094 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:47.591102 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:47.591155 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:47.591278 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:47.592147 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:47.593033 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:47.594084 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:47.594530 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:47.594643 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:47.597786 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:47.597789 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:47.597790 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:47.597801 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:47.599751 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:47.600211 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:47.600214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:47.600215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:47.600330 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:47.600563 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:47.600564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:47.600565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:47.600896 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:47.601050 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:47.601161 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 10:07:47.601328 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":129,"threads_started":1}
I20260808 10:07:47.602591 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:47.612013 5254 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:47.612772 5254 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:47.618402 5254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74ab44c6d2f64e3ba7056e68360623ac: Generated new TSK 0
I20260808 10:07:47.618646 5254 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:47.627787 5248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:47.623618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53494 (local address 127.2.30.190:44257)
0808 10:07:47.623679 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:47.623682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:47.623800 (+ 118us) server_negotiation.cc:407] Connection header received
0808 10:07:47.623848 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:47.623850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:47.623863 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:47.623944 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:47.624426 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:47.625227 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:47.625990 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:47.626345 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:47.626425 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:47.627215 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:47.627217 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:47.627217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:47.627492 (+ 275us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:47.627511 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:47.627519 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:47.627583 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:47.627661 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 10:07:47.627695 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 10:07:47.630344 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 5175
2026-08-08T10:07:47Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1391 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:47 dist-test-slave-w206 krb5kdc[5279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:47 dist-test-slave-w206 krb5kdc[5279](info): set up 2 sockets
Aug 08 10:07:47 dist-test-slave-w206 krb5kdc[5279](info): commencing operation
krb5kdc: starting...
W20260808 10:07:48.703450 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:07:48 dist-test-slave-w206 krb5kdc[5279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:07:48.839052 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:43285
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:45775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:43285
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:48.923580 5293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:48.923840 5293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:48.923882 5293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:48.925141 5293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:48.925216 5293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:48.925240 5293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:48.925261 5293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:48.925292 5293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:48.926836 5293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:45775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:43285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:43285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:48.927142 5293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:48.927398 5293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:48.932447 5298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:48.932505 5299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:48.932417 5301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:48.932709 5293 server_base.cc:1034] running on GCE node
I20260808 10:07:48.932942 5293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:48.933274 5293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:48.934445 5293 hybrid_clock.cc:648] HybridClock initialized: now 1786183668934426 us; error 33 us; skew 500 ppm
Aug 08 10:07:48 dist-test-slave-w206 krb5kdc[5279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183668, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:48.937199 5293 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:07:48.937675 5293 webserver.cc:460] Webserver started at http://127.2.30.190:42573/ using document root <none> and password file <none>
I20260808 10:07:48.937937 5293 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:48.938011 5293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:48.938189 5293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:48.939453 5293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "0f0c9993425a4316929bd8160d2ce0fc"
format_stamp: "Formatted at 2026-08-08 10:07:48 on dist-test-slave-w206"
server_key: "7ab07103f091dc73e7ffe99a6987c169"
server_key_iv: "67ee281040033ed1cce212e470a74295"
server_key_version: "encryptionkey@0"
I20260808 10:07:48.939843 5293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "0f0c9993425a4316929bd8160d2ce0fc"
format_stamp: "Formatted at 2026-08-08 10:07:48 on dist-test-slave-w206"
server_key: "7ab07103f091dc73e7ffe99a6987c169"
server_key_iv: "67ee281040033ed1cce212e470a74295"
server_key_version: "encryptionkey@0"
I20260808 10:07:48.941468 5293 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:07:48.942312 5308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:48.942556 5293 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:48.942689 5293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "0f0c9993425a4316929bd8160d2ce0fc"
format_stamp: "Formatted at 2026-08-08 10:07:48 on dist-test-slave-w206"
server_key: "7ab07103f091dc73e7ffe99a6987c169"
server_key_iv: "67ee281040033ed1cce212e470a74295"
server_key_version: "encryptionkey@0"
I20260808 10:07:48.942778 5293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:48.952564 5293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:48.954823 5293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:48.955072 5293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:48.959128 5293 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:43285
I20260808 10:07:48.959214 5362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:43285 every 8 connection(s)
I20260808 10:07:48.959589 5293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:48.960841 5363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:48.963238 5363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc: Bootstrap starting.
I20260808 10:07:48.963845 5363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:48.964154 5363 log.cc:824] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:48.964879 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 5293
I20260808 10:07:48.964986 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:07:48.965026 5363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc: No bootstrap required, opened a new log
I20260808 10:07:48.965179 2170 external_mini_cluster.cc:1428] Setting key 509a5b29dabbf659cdd5c3b043adeb43
I20260808 10:07:48.966922 5363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f0c9993425a4316929bd8160d2ce0fc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43285 } }
I20260808 10:07:48.967152 5363 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:48.967240 5363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f0c9993425a4316929bd8160d2ce0fc, State: Initialized, Role: FOLLOWER
I20260808 10:07:48.967411 5363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f0c9993425a4316929bd8160d2ce0fc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43285 } }
I20260808 10:07:48.967511 5363 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:48.967558 5363 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:48.967608 5363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:48.968541 5363 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f0c9993425a4316929bd8160d2ce0fc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43285 } }
I20260808 10:07:48.968668 5363 leader_election.cc:302] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f0c9993425a4316929bd8160d2ce0fc; no voters:
I20260808 10:07:48.968888 5363 leader_election.cc:288] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:48.968914 5368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:48.969026 5368 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [term 1 LEADER]: Becoming Leader. State: Replica: 0f0c9993425a4316929bd8160d2ce0fc, State: Running, Role: LEADER
I20260808 10:07:48.969177 5363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:48.969167 5368 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f0c9993425a4316929bd8160d2ce0fc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43285 } }
I20260808 10:07:48.969657 5370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f0c9993425a4316929bd8160d2ce0fc. Latest consensus state: current_term: 1 leader_uuid: "0f0c9993425a4316929bd8160d2ce0fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f0c9993425a4316929bd8160d2ce0fc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43285 } } }
I20260808 10:07:48.969748 5370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:48.969981 5369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f0c9993425a4316929bd8160d2ce0fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f0c9993425a4316929bd8160d2ce0fc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43285 } } }
I20260808 10:07:48.970335 5369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:48.970520 5374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:48.971192 5374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:48.973057 5374 catalog_manager.cc:1347] Generated new cluster ID: 2041fa91921e41fcb8c1013422ea309d
I20260808 10:07:48.973109 5374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:07:48 dist-test-slave-w206 krb5kdc[5279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:48.978483 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:48.965826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60110 (local address 127.2.30.190:43285)
0808 10:07:48.966035 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:48.966043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:48.966071 (+ 28us) server_negotiation.cc:407] Connection header received
0808 10:07:48.966912 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:48.966921 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:48.967030 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:48.967272 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:48.968010 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:48.969121 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:48.970262 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:48.971014 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:48.971122 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:48.974072 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:48.974077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:48.974077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:48.974090 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:48.976246 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:48.976673 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:48.976677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:48.976677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:48.976815 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:48.977039 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:48.977040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:48.977040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:48.977371 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:48.978010 (+ 639us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:48.978088 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 10:07:48.978246 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":91,"threads_started":1}
I20260808 10:07:48.999478 5374 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:49.000284 5374 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:49.007469 5374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f0c9993425a4316929bd8160d2ce0fc: Generated new TSK 0
I20260808 10:07:49.007715 5374 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:07:49.043426 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43285
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:45775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:49.130519 5389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:49.130761 5389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:49.130805 5389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:49.132074 5389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:49.132148 5389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:49.132193 5389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:07:49.133682 5389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:45775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:49.133978 5389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:49.134231 5389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:49.139662 5397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:49.139818 5389 server_base.cc:1034] running on GCE node
W20260808 10:07:49.139680 5394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:49.139946 5395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:49.140204 5389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:49.140491 5389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:49.141717 5389 hybrid_clock.cc:648] HybridClock initialized: now 1786183669141691 us; error 37 us; skew 500 ppm
Aug 08 10:07:49 dist-test-slave-w206 krb5kdc[5279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183669, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:49.144626 5389 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:07:49.145197 5389 webserver.cc:460] Webserver started at http://127.2.30.129:39145/ using document root <none> and password file <none>
I20260808 10:07:49.145475 5389 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:49.145548 5389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:49.145704 5389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:49.146894 5389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "b2705f44d040458eac46804ae2927369"
format_stamp: "Formatted at 2026-08-08 10:07:49 on dist-test-slave-w206"
server_key: "230e85939bd8d883570c8297c97bf862"
server_key_iv: "bb3d06122320c9d63cf0c5f0ae0c1084"
server_key_version: "encryptionkey@0"
I20260808 10:07:49.147356 5389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "b2705f44d040458eac46804ae2927369"
format_stamp: "Formatted at 2026-08-08 10:07:49 on dist-test-slave-w206"
server_key: "230e85939bd8d883570c8297c97bf862"
server_key_iv: "bb3d06122320c9d63cf0c5f0ae0c1084"
server_key_version: "encryptionkey@0"
I20260808 10:07:49.148874 5389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:49.149829 5404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:49.150105 5389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:49.150189 5389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "b2705f44d040458eac46804ae2927369"
format_stamp: "Formatted at 2026-08-08 10:07:49 on dist-test-slave-w206"
server_key: "230e85939bd8d883570c8297c97bf862"
server_key_iv: "bb3d06122320c9d63cf0c5f0ae0c1084"
server_key_version: "encryptionkey@0"
I20260808 10:07:49.150293 5389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:49.174707 5389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:49.177058 5389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:49.177237 5389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:49.177549 5389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:49.177930 5389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:49.177968 5389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:49.177992 5389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:49.178045 5389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:49.184489 5389 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:33537
I20260808 10:07:49.184537 5521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:33537 every 8 connection(s)
I20260808 10:07:49.184924 5389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:07:49.190418 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 5389
I20260808 10:07:49.190562 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:07:49.190789 2170 external_mini_cluster.cc:1428] Setting key 0924afb9b1f2f2a97d26a8bde351d248
Aug 08 10:07:49 dist-test-slave-w206 krb5kdc[5279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183669, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:49.195737 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:49.185810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:49033 (local address 127.2.30.190:43285)
0808 10:07:49.185883 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:49.185885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:49.186142 (+ 257us) server_negotiation.cc:407] Connection header received
0808 10:07:49.186647 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:49.186649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:49.186666 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:49.186752 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:49.187807 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:49.188347 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:49.189479 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:49.189702 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:49.189767 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:49.192249 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:49.192250 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:49.192251 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:49.192262 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:49.193921 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:49.194520 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:49.194521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:49.194522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:49.194583 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:49.194913 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:49.194914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:49.194914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:49.195124 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:49.195234 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:49.195551 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 10:07:49.195652 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:07:49.196022 5524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:49.185776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:43285 (local address 127.2.30.129:49033)
0808 10:07:49.185961 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:49.186008 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:49.186539 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:49.186835 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:49.186838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:49.187494 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:49.187701 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:49.187706 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:49.188453 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:49.188454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:49.189385 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:49.189387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:49.189791 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:49.189792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:49.189899 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:49.190536 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:49.190546 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:49.192150 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:49.194019 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:49.194029 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:49.194032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:49.194417 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:49.194660 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:49.194661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:49.194661 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:49.194817 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:49.195199 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:49.195207 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:49.195391 (+ 184us) client_negotiation.cc:769] Sending connection context
0808 10:07:49.195521 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 10:07:49.195717 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
I20260808 10:07:49.196707 5522 heartbeater.cc:347] Connected to a master server at 127.2.30.190:43285
I20260808 10:07:49.196837 5522 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:49.197077 5522 heartbeater.cc:511] Master 127.2.30.190:43285 requested a full tablet report, sending...
I20260808 10:07:49.197641 5325 ts_manager.cc:194] Registered new tserver with Master: b2705f44d040458eac46804ae2927369 (127.2.30.129:33537)
I20260808 10:07:49.198299 5325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:49033
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:07:49.222865 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43285
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:45775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:49.308681 5527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:49.308950 5527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:49.308991 5527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:49.310171 5527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:49.310258 5527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:49.310302 5527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:07:49.311903 5527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:45775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:49.312259 5527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:49.312510 5527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:07:49.318029 5527 server_base.cc:1034] running on GCE node
W20260808 10:07:49.317979 5535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:49.318063 5533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:49.317968 5532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:49.318545 5527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:49.318832 5527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:49.320007 5527 hybrid_clock.cc:648] HybridClock initialized: now 1786183669320000 us; error 42 us; skew 500 ppm
Aug 08 10:07:49 dist-test-slave-w206 krb5kdc[5279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183669, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:49.322901 5527 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:07:49.323453 5527 webserver.cc:460] Webserver started at http://127.2.30.130:42113/ using document root <none> and password file <none>
I20260808 10:07:49.323731 5527 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:49.323817 5527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:49.323993 5527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:49.325124 5527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "879812a99f8c4e41bb0474e3bf19e199"
format_stamp: "Formatted at 2026-08-08 10:07:49 on dist-test-slave-w206"
server_key: "a5f28289ca62ea8e6ed464fc2e9412d5"
server_key_iv: "b10f08d85a86aac2f27845d85731bc2c"
server_key_version: "encryptionkey@0"
I20260808 10:07:49.325495 5527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "879812a99f8c4e41bb0474e3bf19e199"
format_stamp: "Formatted at 2026-08-08 10:07:49 on dist-test-slave-w206"
server_key: "a5f28289ca62ea8e6ed464fc2e9412d5"
server_key_iv: "b10f08d85a86aac2f27845d85731bc2c"
server_key_version: "encryptionkey@0"
I20260808 10:07:49.326947 5527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:07:49.327840 5542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:49.328053 5527 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:49.328152 5527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "879812a99f8c4e41bb0474e3bf19e199"
format_stamp: "Formatted at 2026-08-08 10:07:49 on dist-test-slave-w206"
server_key: "a5f28289ca62ea8e6ed464fc2e9412d5"
server_key_iv: "b10f08d85a86aac2f27845d85731bc2c"
server_key_version: "encryptionkey@0"
I20260808 10:07:49.328234 5527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:49.348016 5527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:49.350346 5527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:49.350703 5527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:49.351048 5527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:49.351408 5527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:49.351445 5527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:49.351469 5527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:49.351485 5527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:49.357645 5527 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:42635
I20260808 10:07:49.357697 5658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:42635 every 8 connection(s)
I20260808 10:07:49.358047 5527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:07:49.359229 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 5527
I20260808 10:07:49.359350 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:07:49.359540 2170 external_mini_cluster.cc:1428] Setting key 8fd8a8a3e048c0a444fe4ed604be38ff
Aug 08 10:07:49 dist-test-slave-w206 krb5kdc[5279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183669, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
I20260808 10:07:49.369666 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:49.358860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:43939 (local address 127.2.30.190:43285)
0808 10:07:49.358962 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:49.358964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:49.359174 (+ 210us) server_negotiation.cc:407] Connection header received
0808 10:07:49.359865 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:49.359866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:49.359882 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:49.359970 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:49.361258 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:49.361775 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:49.363344 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:49.363553 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:49.363613 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:49.366441 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:49.366443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:49.366443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:49.366451 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:49.368048 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:49.368580 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:49.368582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:49.368582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:49.368647 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:49.368943 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:49.368944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:49.368944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:49.369076 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:49.369161 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:49.369464 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 10:07:49.369565 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 10:07:49.369897 5661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:49.358838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:43285 (local address 127.2.30.130:43939)
0808 10:07:49.359037 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:49.359076 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:49.359731 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:49.360060 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:49.360065 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:49.360920 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:49.361137 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:49.361142 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:49.361898 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:49.361899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:49.363226 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:49.363230 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:49.363652 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:49.363654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:49.363808 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:49.364459 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:49.364470 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:49.366252 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:49.368160 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:49.368169 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:49.368172 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:49.368474 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:49.368723 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:49.368725 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:49.368725 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:49.368854 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:49.369151 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:49.369158 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:49.369321 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 10:07:49.369443 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 10:07:49.369620 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":87,"threads_started":1}
I20260808 10:07:49.370572 5659 heartbeater.cc:347] Connected to a master server at 127.2.30.190:43285
I20260808 10:07:49.370708 5659 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:49.370910 5659 heartbeater.cc:511] Master 127.2.30.190:43285 requested a full tablet report, sending...
I20260808 10:07:49.371407 5325 ts_manager.cc:194] Registered new tserver with Master: 879812a99f8c4e41bb0474e3bf19e199 (127.2.30.130:42635)
I20260808 10:07:49.371796 5325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:43939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:07:49.390803 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43285
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:45775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:49.477169 5664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:49.477401 5664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:49.477427 5664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:49.478663 5664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:49.478713 5664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:49.478740 5664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:07:49.480399 5664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:45775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:49.480671 5664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:49.480940 5664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:49.486371 5669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:49.486394 5670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:49.486443 5673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:49.486773 5664 server_base.cc:1034] running on GCE node
I20260808 10:07:49.487111 5664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:49.487346 5664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:49.488525 5664 hybrid_clock.cc:648] HybridClock initialized: now 1786183669488521 us; error 41 us; skew 500 ppm
Aug 08 10:07:49 dist-test-slave-w206 krb5kdc[5279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183669, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:49.491376 5664 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:07:49.491922 5664 webserver.cc:460] Webserver started at http://127.2.30.131:43705/ using document root <none> and password file <none>
I20260808 10:07:49.492172 5664 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:49.492244 5664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:49.492448 5664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:49.493577 5664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "48b44cdc4ee74cea9889f62a9bd33d91"
format_stamp: "Formatted at 2026-08-08 10:07:49 on dist-test-slave-w206"
server_key: "ce99b150224f115cfdd55c620713c49e"
server_key_iv: "d88c39f9686f9f56085d95c6c9ffb0f6"
server_key_version: "encryptionkey@0"
I20260808 10:07:49.493974 5664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "48b44cdc4ee74cea9889f62a9bd33d91"
format_stamp: "Formatted at 2026-08-08 10:07:49 on dist-test-slave-w206"
server_key: "ce99b150224f115cfdd55c620713c49e"
server_key_iv: "d88c39f9686f9f56085d95c6c9ffb0f6"
server_key_version: "encryptionkey@0"
I20260808 10:07:49.495451 5664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:07:49.496378 5680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:49.496637 5664 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:49.496757 5664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "48b44cdc4ee74cea9889f62a9bd33d91"
format_stamp: "Formatted at 2026-08-08 10:07:49 on dist-test-slave-w206"
server_key: "ce99b150224f115cfdd55c620713c49e"
server_key_iv: "d88c39f9686f9f56085d95c6c9ffb0f6"
server_key_version: "encryptionkey@0"
I20260808 10:07:49.496867 5664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:49.527964 5664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:49.530406 5664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:49.530601 5664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:49.530879 5664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:49.531281 5664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:49.531345 5664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:49.531389 5664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:49.531440 5664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:49.537925 5664 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:39743
I20260808 10:07:49.537972 5798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:39743 every 8 connection(s)
I20260808 10:07:49.538337 5664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
Aug 08 10:07:49 dist-test-slave-w206 krb5kdc[5279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183669, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:49.547541 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 5664
I20260808 10:07:49.547669 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:07:49.547890 2170 external_mini_cluster.cc:1428] Setting key e4b39b7a08653b76d7ff76482d39eeb4
I20260808 10:07:49.548498 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:49.539227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:36573 (local address 127.2.30.190:43285)
0808 10:07:49.539310 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:49.539312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:49.539473 (+ 161us) server_negotiation.cc:407] Connection header received
0808 10:07:49.539997 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:49.539998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:49.540009 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:49.540066 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:49.541046 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:49.541566 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:49.542694 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:49.542886 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:49.542940 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:49.545376 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:49.545378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:49.545378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:49.545386 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:49.546909 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:49.547375 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:49.547377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:49.547378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:49.547438 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:49.547701 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:49.547702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:49.547702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:49.547865 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:49.547941 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:49.548305 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 10:07:49.548396 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:07:49.548761 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:49.539174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:43285 (local address 127.2.30.131:36573)
0808 10:07:49.539346 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:49.539385 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:49.539881 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:49.540122 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:49.540126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:49.540745 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:49.540955 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:49.540960 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:49.541686 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:49.541687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:49.542600 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:49.542603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:49.543049 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:49.543051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:49.543173 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:49.543726 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:49.543735 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:49.545211 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:49.547015 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:49.547024 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:49.547027 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:49.547269 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:49.547512 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:49.547513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:49.547513 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:49.547626 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:49.547943 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:49.547951 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:49.548148 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 10:07:49.548268 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 10:07:49.548505 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":105,"threads_started":1}
I20260808 10:07:49.549360 5799 heartbeater.cc:347] Connected to a master server at 127.2.30.190:43285
I20260808 10:07:49.549458 5799 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:49.549667 5799 heartbeater.cc:511] Master 127.2.30.190:43285 requested a full tablet report, sending...
I20260808 10:07:49.550228 5325 ts_manager.cc:194] Registered new tserver with Master: 48b44cdc4ee74cea9889f62a9bd33d91 (127.2.30.131:39743)
I20260808 10:07:49.550716 5325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:36573
I20260808 10:07:49.560468 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:07:49 dist-test-slave-w206 krb5kdc[5279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183669, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 10:07:49 dist-test-slave-w206 krb5kdc[5279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183669, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:49.586751 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:49.579027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60114 (local address 127.2.30.190:43285)
0808 10:07:49.579107 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:49.579109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:49.579131 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:07:49.579265 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:49.579266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:49.579277 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:49.579333 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:49.579974 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:49.580501 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:49.581288 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:49.581478 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:49.581534 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:49.583967 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:49.583968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:49.583968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:49.583976 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:49.585366 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:49.585777 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:49.585778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:49.585778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:49.585831 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:49.586017 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:49.586017 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:49.586017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:49.586176 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:49.586329 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:49.586498 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 10:07:49.586621 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 10:07:49.587138 5325 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60114
I20260808 10:07:49.587745 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 5389
I20260808 10:07:49.593945 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 5527
I20260808 10:07:49.599351 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 5664
I20260808 10:07:49.606842 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 5293
2026-08-08T10:07:49Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1978 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:49 dist-test-slave-w206 krb5kdc[5812](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:49 dist-test-slave-w206 krb5kdc[5812](info): set up 2 sockets
Aug 08 10:07:49 dist-test-slave-w206 krb5kdc[5812](info): commencing operation
krb5kdc: starting...
W20260808 10:07:50.691941 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:07:50 dist-test-slave-w206 krb5kdc[5812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:07:50.826633 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:41979
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:39985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:41979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:50.911108 5826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:50.911378 5826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:50.911422 5826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:50.912629 5826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:50.912705 5826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:50.912731 5826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:50.912770 5826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:50.912791 5826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:50.914340 5826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:41979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:41979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:50.914630 5826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:50.914875 5826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:50.920174 5832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:50.920264 5834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:50.920244 5831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:50.920244 5826 server_base.cc:1034] running on GCE node
I20260808 10:07:50.920662 5826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:50.920943 5826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:50.922133 5826 hybrid_clock.cc:648] HybridClock initialized: now 1786183670922098 us; error 49 us; skew 500 ppm
Aug 08 10:07:50 dist-test-slave-w206 krb5kdc[5812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183670, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:50.925064 5826 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:07:50.925576 5826 webserver.cc:460] Webserver started at http://127.2.30.190:36023/ using document root <none> and password file <none>
I20260808 10:07:50.925931 5826 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:50.926024 5826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:50.926192 5826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:50.927405 5826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "964113908d084292959dfc7ac45e9d04"
format_stamp: "Formatted at 2026-08-08 10:07:50 on dist-test-slave-w206"
server_key: "c9f4f681e9a87285e433ab576956c135"
server_key_iv: "e7337ac2da0203031aefe10b22bf72f0"
server_key_version: "encryptionkey@0"
I20260808 10:07:50.927806 5826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "964113908d084292959dfc7ac45e9d04"
format_stamp: "Formatted at 2026-08-08 10:07:50 on dist-test-slave-w206"
server_key: "c9f4f681e9a87285e433ab576956c135"
server_key_iv: "e7337ac2da0203031aefe10b22bf72f0"
server_key_version: "encryptionkey@0"
I20260808 10:07:50.929378 5826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:07:50.930223 5841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:50.930435 5826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:50.930533 5826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "964113908d084292959dfc7ac45e9d04"
format_stamp: "Formatted at 2026-08-08 10:07:50 on dist-test-slave-w206"
server_key: "c9f4f681e9a87285e433ab576956c135"
server_key_iv: "e7337ac2da0203031aefe10b22bf72f0"
server_key_version: "encryptionkey@0"
I20260808 10:07:50.930619 5826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:50.950199 5826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:50.952280 5826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:50.952471 5826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:50.956946 5826 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:41979
I20260808 10:07:50.956983 5895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:41979 every 8 connection(s)
I20260808 10:07:50.957571 5826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:50.958380 5897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:50.960853 5897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04: Bootstrap starting.
I20260808 10:07:50.961431 5897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:50.961686 5897 log.cc:824] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:50.962404 5897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04: No bootstrap required, opened a new log
I20260808 10:07:50.963126 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 5826
I20260808 10:07:50.963234 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:07:50.963433 2170 external_mini_cluster.cc:1428] Setting key e3dedcabc38258afce19817d437ceb1f
I20260808 10:07:50.965373 5897 raft_consensus.cc:348] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "964113908d084292959dfc7ac45e9d04" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41979 } }
I20260808 10:07:50.965553 5897 raft_consensus.cc:374] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:50.965591 5897 raft_consensus.cc:729] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964113908d084292959dfc7ac45e9d04, State: Initialized, Role: FOLLOWER
I20260808 10:07:50.965785 5897 consensus_queue.cc:260] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "964113908d084292959dfc7ac45e9d04" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41979 } }
I20260808 10:07:50.965866 5897 raft_consensus.cc:388] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:50.965890 5897 raft_consensus.cc:482] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:50.965939 5897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:50.966641 5897 raft_consensus.cc:504] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "964113908d084292959dfc7ac45e9d04" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41979 } }
I20260808 10:07:50.966751 5897 leader_election.cc:302] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 964113908d084292959dfc7ac45e9d04; no voters:
I20260808 10:07:50.966931 5897 leader_election.cc:288] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:50.967068 5903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:50.967265 5897 sys_catalog.cc:564] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:50.967281 5903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [term 1 LEADER]: Becoming Leader. State: Replica: 964113908d084292959dfc7ac45e9d04, State: Running, Role: LEADER
I20260808 10:07:50.967494 5903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "964113908d084292959dfc7ac45e9d04" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41979 } }
I20260808 10:07:50.967862 5904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "964113908d084292959dfc7ac45e9d04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "964113908d084292959dfc7ac45e9d04" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41979 } } }
I20260808 10:07:50.967945 5904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:50.968132 5905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 964113908d084292959dfc7ac45e9d04. Latest consensus state: current_term: 1 leader_uuid: "964113908d084292959dfc7ac45e9d04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "964113908d084292959dfc7ac45e9d04" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41979 } } }
I20260808 10:07:50.968235 5905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:50.968721 5913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:50.969311 5913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:50.970950 5913 catalog_manager.cc:1347] Generated new cluster ID: 9f260b4cc1914919b3d495185638d809
I20260808 10:07:50.971031 5913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:07:50 dist-test-slave-w206 krb5kdc[5812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:50.977085 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:50.963977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37614 (local address 127.2.30.190:41979)
0808 10:07:50.964134 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:50.964144 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:50.964178 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:07:50.965320 (+ 1142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:50.965330 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:50.965398 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:50.965567 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:50.966379 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:50.967838 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:50.969024 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:50.969638 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:50.969754 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:50.972726 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:50.972731 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:50.972731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:50.972745 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:50.974895 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:50.975339 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:50.975343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:50.975343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:50.975463 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:50.975672 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:50.975673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:50.975673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:50.976001 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:50.976182 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:50.976700 (+ 518us) server_negotiation.cc:299] Negotiation successful
0808 10:07:50.976862 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260808 10:07:50.982474 5913 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:50.983093 5913 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:50.992255 5913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 964113908d084292959dfc7ac45e9d04: Generated new TSK 0
I20260808 10:07:50.992493 5913 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:07:51.027854 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41979
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:39985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:51.112563 5926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:51.112802 5926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:51.112828 5926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:51.114099 5926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:51.114146 5926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:51.114173 5926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:07:51.115780 5926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:51.116057 5926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:51.116312 5926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:07:51.121685 5926 server_base.cc:1034] running on GCE node
W20260808 10:07:51.121636 5936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:51.121848 5932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:51.121654 5934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:51.122123 5926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:51.122341 5926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:51.123485 5926 hybrid_clock.cc:648] HybridClock initialized: now 1786183671123475 us; error 29 us; skew 500 ppm
Aug 08 10:07:51 dist-test-slave-w206 krb5kdc[5812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:51.126518 5926 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:07:51.127029 5926 webserver.cc:460] Webserver started at http://127.2.30.129:43669/ using document root <none> and password file <none>
I20260808 10:07:51.127287 5926 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:51.127359 5926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:51.127523 5926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:51.128820 5926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "7b3cf652cdc14957a35a6d73a3baa5ad"
format_stamp: "Formatted at 2026-08-08 10:07:51 on dist-test-slave-w206"
server_key: "5eddfcc4d4b5a2cf6ffac4bf03dafebc"
server_key_iv: "46ec8aec449faee482c6fa02cac00d82"
server_key_version: "encryptionkey@0"
I20260808 10:07:51.129196 5926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "7b3cf652cdc14957a35a6d73a3baa5ad"
format_stamp: "Formatted at 2026-08-08 10:07:51 on dist-test-slave-w206"
server_key: "5eddfcc4d4b5a2cf6ffac4bf03dafebc"
server_key_iv: "46ec8aec449faee482c6fa02cac00d82"
server_key_version: "encryptionkey@0"
I20260808 10:07:51.130601 5926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:07:51.131450 5944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:51.131677 5926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:07:51.131824 5926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "7b3cf652cdc14957a35a6d73a3baa5ad"
format_stamp: "Formatted at 2026-08-08 10:07:51 on dist-test-slave-w206"
server_key: "5eddfcc4d4b5a2cf6ffac4bf03dafebc"
server_key_iv: "46ec8aec449faee482c6fa02cac00d82"
server_key_version: "encryptionkey@0"
I20260808 10:07:51.131911 5926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:51.144282 5926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:51.147128 5926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:51.147324 5926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:51.147593 5926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:51.147961 5926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:51.148026 5926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:51.148073 5926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:51.148102 5926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:51.154786 5926 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:37811
I20260808 10:07:51.154834 6064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:37811 every 8 connection(s)
I20260808 10:07:51.155277 5926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
Aug 08 10:07:51 dist-test-slave-w206 krb5kdc[5812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:51.164193 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 5926
I20260808 10:07:51.164296 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:07:51.164574 2170 external_mini_cluster.cc:1428] Setting key 74f7d6eefe9f88e545d0ee9529f0d496
I20260808 10:07:51.166764 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:51.156152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:36491 (local address 127.2.30.190:41979)
0808 10:07:51.156229 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:51.156232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:51.156531 (+ 299us) server_negotiation.cc:407] Connection header received
0808 10:07:51.157493 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:51.157494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:51.157507 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:51.157572 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:51.158263 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:51.158819 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:51.160060 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:51.160330 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:51.160400 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:51.163139 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:51.163140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:51.163141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:51.163149 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:51.165087 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:51.165613 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:51.165615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:51.165615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:51.165688 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:51.165955 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:51.165956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:51.165956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:51.166118 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:51.166193 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:51.166480 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 10:07:51.166590 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:07:51.166792 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:51.156140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41979 (local address 127.2.30.129:36491)
0808 10:07:51.156393 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:51.156436 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:51.157378 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:51.157627 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:51.157631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:51.157927 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:51.158142 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:51.158147 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:51.158935 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:51.158936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:51.159946 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:51.159949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:51.160446 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:51.160447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:51.160574 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:51.161268 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:51.161277 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:51.163036 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:51.165172 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:51.165182 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:51.165184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:51.165518 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:51.165760 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:51.165760 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:51.165761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:51.165885 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:51.166188 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:51.166194 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:51.166340 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 10:07:51.166432 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 10:07:51.166564 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":134,"threads_started":1}
I20260808 10:07:51.167411 6066 heartbeater.cc:347] Connected to a master server at 127.2.30.190:41979
I20260808 10:07:51.167541 6066 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:51.167852 6066 heartbeater.cc:511] Master 127.2.30.190:41979 requested a full tablet report, sending...
I20260808 10:07:51.168536 5859 ts_manager.cc:194] Registered new tserver with Master: 7b3cf652cdc14957a35a6d73a3baa5ad (127.2.30.129:37811)
I20260808 10:07:51.169525 5859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:36491
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:07:51.194849 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41979
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:39985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:51.281267 6071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:51.281483 6071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:51.281510 6071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:51.282708 6071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:51.282759 6071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:51.282788 6071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:07:51.284389 6071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:51.284662 6071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:51.284912 6071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:51.290320 6079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:51.290336 6076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:51.290447 6077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:51.290781 6071 server_base.cc:1034] running on GCE node
I20260808 10:07:51.291112 6071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:51.291399 6071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:51.292568 6071 hybrid_clock.cc:648] HybridClock initialized: now 1786183671292538 us; error 49 us; skew 500 ppm
Aug 08 10:07:51 dist-test-slave-w206 krb5kdc[5812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:51.295518 6071 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:07:51.296054 6071 webserver.cc:460] Webserver started at http://127.2.30.130:39247/ using document root <none> and password file <none>
I20260808 10:07:51.296350 6071 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:51.296434 6071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:51.296623 6071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:51.297803 6071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "ed468848a82b4efb8ea9cb10ef50636a"
format_stamp: "Formatted at 2026-08-08 10:07:51 on dist-test-slave-w206"
server_key: "672149fadef77877f8e615e29637cfc9"
server_key_iv: "400b59f787a8d47534605ebb17705ae7"
server_key_version: "encryptionkey@0"
I20260808 10:07:51.298219 6071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "ed468848a82b4efb8ea9cb10ef50636a"
format_stamp: "Formatted at 2026-08-08 10:07:51 on dist-test-slave-w206"
server_key: "672149fadef77877f8e615e29637cfc9"
server_key_iv: "400b59f787a8d47534605ebb17705ae7"
server_key_version: "encryptionkey@0"
I20260808 10:07:51.299824 6071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:07:51.300742 6087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:51.300927 6071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:07:51.301036 6071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "ed468848a82b4efb8ea9cb10ef50636a"
format_stamp: "Formatted at 2026-08-08 10:07:51 on dist-test-slave-w206"
server_key: "672149fadef77877f8e615e29637cfc9"
server_key_iv: "400b59f787a8d47534605ebb17705ae7"
server_key_version: "encryptionkey@0"
I20260808 10:07:51.301124 6071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:51.314162 6071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:51.316306 6071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:51.316480 6071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:51.316736 6071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:51.317063 6071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:51.317119 6071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:51.317155 6071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:51.317206 6071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:51.323045 6071 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:40567
I20260808 10:07:51.323087 6203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:40567 every 8 connection(s)
I20260808 10:07:51.323444 6071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
Aug 08 10:07:51 dist-test-slave-w206 krb5kdc[5812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:51.330856 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 6071
I20260808 10:07:51.331023 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:07:51.331355 2170 external_mini_cluster.cc:1428] Setting key 4d0b63d0f4dd525dd2cc3fc8bc1de5e3
I20260808 10:07:51.334056 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:51.324338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:40669 (local address 127.2.30.190:41979)
0808 10:07:51.324440 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:51.324443 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:51.324615 (+ 172us) server_negotiation.cc:407] Connection header received
0808 10:07:51.325618 (+ 1003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:51.325619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:51.325632 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:51.325706 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:51.326505 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:51.327138 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:51.328214 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:51.328431 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:51.328512 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:51.330724 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:51.330726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:51.330726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:51.330734 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:51.332395 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:51.332949 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:51.332950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:51.332950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:51.333004 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:51.333310 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:51.333311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:51.333311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:51.333415 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:51.333507 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:51.333853 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 10:07:51.333970 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:07:51.334348 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:51.324235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41979 (local address 127.2.30.130:40669)
0808 10:07:51.324485 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:51.324523 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:51.325499 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:51.325789 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:51.325792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:51.326082 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:51.326395 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:51.326406 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:51.327242 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:51.327244 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:51.328113 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:51.328116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:51.328518 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:51.328520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:51.328648 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:51.329240 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:51.329248 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:51.330617 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:51.332484 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:51.332492 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:51.332495 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:51.332867 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:51.333077 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:51.333078 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:51.333078 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:51.333239 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:51.333498 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:51.333505 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:51.333692 (+ 187us) client_negotiation.cc:769] Sending connection context
0808 10:07:51.333827 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 10:07:51.334028 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":96,"threads_started":1}
I20260808 10:07:51.335055 6204 heartbeater.cc:347] Connected to a master server at 127.2.30.190:41979
I20260808 10:07:51.335188 6204 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:51.335446 6204 heartbeater.cc:511] Master 127.2.30.190:41979 requested a full tablet report, sending...
I20260808 10:07:51.335901 5859 ts_manager.cc:194] Registered new tserver with Master: ed468848a82b4efb8ea9cb10ef50636a (127.2.30.130:40567)
I20260808 10:07:51.336292 5859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:40669
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:07:51.362563 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41979
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:39985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:51.451447 6209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:51.451689 6209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:51.451733 6209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:51.452903 6209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:51.452972 6209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:51.453019 6209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:07:51.454640 6209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:51.454923 6209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:51.455183 6209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:07:51.460497 6209 server_base.cc:1034] running on GCE node
W20260808 10:07:51.460479 6215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:51.460441 6217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:51.460443 6214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:51.460867 6209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:51.461120 6209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:51.462296 6209 hybrid_clock.cc:648] HybridClock initialized: now 1786183671462269 us; error 43 us; skew 500 ppm
Aug 08 10:07:51 dist-test-slave-w206 krb5kdc[5812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:51.464954 6209 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:07:51.465385 6209 webserver.cc:460] Webserver started at http://127.2.30.131:34665/ using document root <none> and password file <none>
I20260808 10:07:51.465613 6209 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:51.465682 6209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:51.465837 6209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:51.466886 6209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "5476815dacd94f8594e45233f36d4b32"
format_stamp: "Formatted at 2026-08-08 10:07:51 on dist-test-slave-w206"
server_key: "d947fd45a1b15e63be9302011ad3e788"
server_key_iv: "1184f0877a0d93d141d30d694a0717d8"
server_key_version: "encryptionkey@0"
I20260808 10:07:51.467336 6209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "5476815dacd94f8594e45233f36d4b32"
format_stamp: "Formatted at 2026-08-08 10:07:51 on dist-test-slave-w206"
server_key: "d947fd45a1b15e63be9302011ad3e788"
server_key_iv: "1184f0877a0d93d141d30d694a0717d8"
server_key_version: "encryptionkey@0"
I20260808 10:07:51.469065 6209 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:07:51.469846 6224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:51.470036 6209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:51.470137 6209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "5476815dacd94f8594e45233f36d4b32"
format_stamp: "Formatted at 2026-08-08 10:07:51 on dist-test-slave-w206"
server_key: "d947fd45a1b15e63be9302011ad3e788"
server_key_iv: "1184f0877a0d93d141d30d694a0717d8"
server_key_version: "encryptionkey@0"
I20260808 10:07:51.470227 6209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:51.482795 6209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:51.485234 6209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:51.485416 6209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:51.485673 6209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:51.486003 6209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:51.486059 6209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:51.486104 6209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:51.486129 6209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:51.492673 6209 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:33815
I20260808 10:07:51.492779 6341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:33815 every 8 connection(s)
I20260808 10:07:51.493088 6209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:07:51.498540 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 6209
I20260808 10:07:51.498656 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:07:51.498929 2170 external_mini_cluster.cc:1428] Setting key f36dd76f8b9b744994b9282b30f9cda2
Aug 08 10:07:51 dist-test-slave-w206 krb5kdc[5812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183671, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:51.503501 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:51.493892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:40223 (local address 127.2.30.190:41979)
0808 10:07:51.493935 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:51.493937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:51.494183 (+ 246us) server_negotiation.cc:407] Connection header received
0808 10:07:51.495368 (+ 1185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:51.495369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:51.495379 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:51.495439 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:51.496052 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:51.496587 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:51.497696 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:51.497888 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:51.497942 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:51.500608 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:51.500609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:51.500609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:51.500616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:51.502092 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:51.502491 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:51.502491 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:51.502492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:51.502545 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:51.502778 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:51.502778 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:51.502779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:51.502888 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:51.502943 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:51.503358 (+ 415us) server_negotiation.cc:299] Negotiation successful
0808 10:07:51.503433 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:07:51.503679 6344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:51.493867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41979 (local address 127.2.30.131:40223)
0808 10:07:51.494045 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:51.494090 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:51.495273 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:51.495498 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:51.495502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:51.495773 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:51.495979 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:51.495982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:51.496679 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:51.496680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:51.497617 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:51.497619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:51.497971 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:51.497972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:51.498086 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:51.498647 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:51.498658 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:51.500519 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:51.502181 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:51.502188 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:51.502190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:51.502409 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:51.502610 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:51.502611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:51.502611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:51.502720 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:51.503042 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:51.503050 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:51.503200 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 10:07:51.503314 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 10:07:51.503458 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
I20260808 10:07:51.504230 6342 heartbeater.cc:347] Connected to a master server at 127.2.30.190:41979
I20260808 10:07:51.504352 6342 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:51.504549 6342 heartbeater.cc:511] Master 127.2.30.190:41979 requested a full tablet report, sending...
I20260808 10:07:51.504905 5859 ts_manager.cc:194] Registered new tserver with Master: 5476815dacd94f8594e45233f36d4b32 (127.2.30.131:33815)
I20260808 10:07:51.505347 5859 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:40223
I20260808 10:07:51.511124 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 10:07:51 dist-test-slave-w206 krb5kdc[5812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:07:51 dist-test-slave-w206 krb5kdc[5812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.129@KRBTEST.COM
I20260808 10:07:51.539449 6349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:51.530691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56914 (local address 127.2.30.129:37811)
0808 10:07:51.530914 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:51.530920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:51.530935 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:07:51.531058 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:51.531062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:51.531084 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:51.531176 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:51.531806 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:51.532569 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:51.533411 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:51.533760 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:51.533849 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:51.536433 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:51.536441 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:51.536441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:51.536449 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:51.538161 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:51.538538 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:51.538541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:51.538541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:51.538599 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:51.538859 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:51.538859 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:51.538860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:51.539040 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:51.539166 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:51.539292 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 10:07:51.539356 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":92,"threads_started":1}
Aug 08 10:07:51 dist-test-slave-w206 krb5kdc[5812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:51.548310 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:51.540897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37626 (local address 127.2.30.190:41979)
0808 10:07:51.540972 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:51.540974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:51.541012 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:07:51.541102 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:51.541102 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:51.541113 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:51.541168 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:51.541651 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:51.542151 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:51.542828 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:51.543104 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:51.543189 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:51.545584 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:51.545585 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:51.545586 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:51.545592 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:51.547134 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:51.547529 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:51.547530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:51.547530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:51.547585 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:51.547783 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:51.547784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:51.547784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:51.547931 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:51.548011 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:51.548173 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 10:07:51.548242 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 10:07:51.548624 5859 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37626
I20260808 10:07:51.549041 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 5926
I20260808 10:07:51.555857 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 6071
I20260808 10:07:51.562350 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 6209
I20260808 10:07:51.568702 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 5826
2026-08-08T10:07:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1961 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:51 dist-test-slave-w206 krb5kdc[6354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:51 dist-test-slave-w206 krb5kdc[6354](info): set up 2 sockets
Aug 08 10:07:51 dist-test-slave-w206 krb5kdc[6354](info): commencing operation
krb5kdc: starting...
W20260808 10:07:52.670840 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:07:52 dist-test-slave-w206 krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:07:52.807560 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:40889
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:35465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:40889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:52.896183 6368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:52.896405 6368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:52.896430 6368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:52.897634 6368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:52.897683 6368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:52.897697 6368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:52.897710 6368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:52.897723 6368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:52.899392 6368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:40889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:40889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:52.899669 6368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:52.899964 6368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:52.905553 6374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:52.905622 6377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:52.905843 6368 server_base.cc:1034] running on GCE node
W20260808 10:07:52.905580 6375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:52.906234 6368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:52.906515 6368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:52.907685 6368 hybrid_clock.cc:648] HybridClock initialized: now 1786183672907667 us; error 33 us; skew 500 ppm
Aug 08 10:07:52 dist-test-slave-w206 krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183672, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:52.918753 6368 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:07:52.919260 6368 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:52.919370 6368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:52.919497 6368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:52.920953 6368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "8f27a499e22242519b1d492959e6efe2"
format_stamp: "Formatted at 2026-08-08 10:07:52 on dist-test-slave-w206"
server_key: "46e9a8e68899962a6f2d761fe378d9f3"
server_key_iv: "6d40fccee9f8d6867531fb50c895c9d0"
server_key_version: "encryptionkey@0"
I20260808 10:07:52.921402 6368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "8f27a499e22242519b1d492959e6efe2"
format_stamp: "Formatted at 2026-08-08 10:07:52 on dist-test-slave-w206"
server_key: "46e9a8e68899962a6f2d761fe378d9f3"
server_key_iv: "6d40fccee9f8d6867531fb50c895c9d0"
server_key_version: "encryptionkey@0"
I20260808 10:07:52.922842 6368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:07:52.923862 6384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:52.924052 6368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:52.924158 6368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "8f27a499e22242519b1d492959e6efe2"
format_stamp: "Formatted at 2026-08-08 10:07:52 on dist-test-slave-w206"
server_key: "46e9a8e68899962a6f2d761fe378d9f3"
server_key_iv: "6d40fccee9f8d6867531fb50c895c9d0"
server_key_version: "encryptionkey@0"
I20260808 10:07:52.924220 6368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:52.941311 6368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:52.943744 6368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:52.943881 6368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:52.948179 6368 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:40889
I20260808 10:07:52.948207 6436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:40889 every 8 connection(s)
I20260808 10:07:52.948567 6368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:52.949625 6437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:52.952648 6437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2: Bootstrap starting.
I20260808 10:07:52.953336 6437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:52.953658 6437 log.cc:824] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:52.954457 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 6368
I20260808 10:07:52.954493 6437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2: No bootstrap required, opened a new log
I20260808 10:07:52.954553 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:07:52.954758 2170 external_mini_cluster.cc:1428] Setting key 6cc382cca2b3bc0045075c35c952f3d9
I20260808 10:07:52.956845 6437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f27a499e22242519b1d492959e6efe2" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 40889 } }
I20260808 10:07:52.957036 6437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:52.957137 6437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f27a499e22242519b1d492959e6efe2, State: Initialized, Role: FOLLOWER
I20260808 10:07:52.957361 6437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f27a499e22242519b1d492959e6efe2" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 40889 } }
I20260808 10:07:52.957474 6437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:52.957508 6437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:52.957639 6437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:52.958560 6437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f27a499e22242519b1d492959e6efe2" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 40889 } }
I20260808 10:07:52.958680 6437 leader_election.cc:302] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f27a499e22242519b1d492959e6efe2; no voters:
I20260808 10:07:52.958861 6437 leader_election.cc:288] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:52.958938 6442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:52.959111 6442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [term 1 LEADER]: Becoming Leader. State: Replica: 8f27a499e22242519b1d492959e6efe2, State: Running, Role: LEADER
I20260808 10:07:52.959259 6442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f27a499e22242519b1d492959e6efe2" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 40889 } }
I20260808 10:07:52.959141 6437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:52.959638 6444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f27a499e22242519b1d492959e6efe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f27a499e22242519b1d492959e6efe2" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 40889 } } }
I20260808 10:07:52.959645 6443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f27a499e22242519b1d492959e6efe2. Latest consensus state: current_term: 1 leader_uuid: "8f27a499e22242519b1d492959e6efe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f27a499e22242519b1d492959e6efe2" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 40889 } } }
I20260808 10:07:52.959754 6443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:52.959748 6444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2 [sys.catalog]: This master's current role is: LEADER
W20260808 10:07:52.961609 6460 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:07:52.961723 6460 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:07:52.961810 6446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:52.961998 6446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:52.963749 6446 catalog_manager.cc:1347] Generated new cluster ID: d908c4aea78344199e23cbe29be8fb8f
I20260808 10:07:52.963809 6446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:07:52 dist-test-slave-w206 krb5kdc[6354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:52.968531 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:52.955432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33494 (local address 127.2.30.190:40889)
0808 10:07:52.955655 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:52.955665 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:52.955707 (+ 42us) server_negotiation.cc:407] Connection header received
0808 10:07:52.956801 (+ 1094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:52.956810 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:52.956881 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:52.957043 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:52.958020 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:52.959479 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:52.960717 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:52.961487 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:52.961603 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:52.964527 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:52.964532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:52.964532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:52.964545 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:52.966708 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:52.967286 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:52.967289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:52.967289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:52.967392 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:52.967629 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:52.967630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:52.967630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:52.967895 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:52.968098 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:52.968185 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 10:07:52.968350 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":105,"threads_started":1}
I20260808 10:07:52.977110 6446 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:52.977867 6446 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:52.986698 6446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2: Generated new TSK 0
I20260808 10:07:52.987035 6446 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:07:53.029665 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:40889
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:35465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:53.118911 6465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:53.119169 6465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:53.119203 6465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:53.120436 6465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:53.120481 6465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:53.120509 6465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:07:53.122043 6465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:40889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:53.122260 6465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:53.122460 6465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:53.127889 6473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:53.127940 6471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:53.127941 6470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:53.128072 6465 server_base.cc:1034] running on GCE node
I20260808 10:07:53.128584 6465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:53.128875 6465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:53.130064 6465 hybrid_clock.cc:648] HybridClock initialized: now 1786183673130052 us; error 60 us; skew 500 ppm
Aug 08 10:07:53 dist-test-slave-w206 krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183673, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:53.133414 6465 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:07:53.133816 6465 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:53.133908 6465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:53.134066 6465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:53.135336 6465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "289b08fb924f43a9882768df51b3fa1e"
format_stamp: "Formatted at 2026-08-08 10:07:53 on dist-test-slave-w206"
server_key: "8979d43ebd37ad632bd0fb4e59cf600e"
server_key_iv: "2dd8b2ba8824a4b3a25671e0680b0736"
server_key_version: "encryptionkey@0"
I20260808 10:07:53.135756 6465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "289b08fb924f43a9882768df51b3fa1e"
format_stamp: "Formatted at 2026-08-08 10:07:53 on dist-test-slave-w206"
server_key: "8979d43ebd37ad632bd0fb4e59cf600e"
server_key_iv: "2dd8b2ba8824a4b3a25671e0680b0736"
server_key_version: "encryptionkey@0"
I20260808 10:07:53.137197 6465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:07:53.138036 6479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:53.138269 6465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:53.138404 6465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "289b08fb924f43a9882768df51b3fa1e"
format_stamp: "Formatted at 2026-08-08 10:07:53 on dist-test-slave-w206"
server_key: "8979d43ebd37ad632bd0fb4e59cf600e"
server_key_iv: "2dd8b2ba8824a4b3a25671e0680b0736"
server_key_version: "encryptionkey@0"
I20260808 10:07:53.138497 6465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:53.158934 6465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:53.162163 6465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:53.162352 6465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:53.162638 6465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:53.163077 6465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:53.163136 6465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:53.163180 6465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:53.163213 6465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:53.169360 6465 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:43515
I20260808 10:07:53.169399 6597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:43515 every 8 connection(s)
I20260808 10:07:53.169713 6465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:07:53.176312 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 6465
I20260808 10:07:53.176419 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
Aug 08 10:07:53 dist-test-slave-w206 krb5kdc[6354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183673, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:53.176638 2170 external_mini_cluster.cc:1428] Setting key a353fe14971d874901fad16473e54a24
I20260808 10:07:53.180874 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.170520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:57585 (local address 127.2.30.190:40889)
0808 10:07:53.170690 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:53.170692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:53.170908 (+ 216us) server_negotiation.cc:407] Connection header received
0808 10:07:53.171496 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:53.171497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:53.171515 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:53.171575 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:53.172594 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.173115 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.174171 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.174400 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.174484 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.177141 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:53.177143 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:53.177143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:53.177155 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:53.179129 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:53.179673 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:53.179675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:53.179676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:53.179764 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:53.180079 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:53.180080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:53.180080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:53.180215 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:53.180301 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:53.180602 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 10:07:53.180768 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260808 10:07:53.181079 6600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.170490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:40889 (local address 127.2.30.129:57585)
0808 10:07:53.170728 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:53.170779 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:53.171367 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:53.171654 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:53.171658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:53.171951 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:53.172480 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:53.172486 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.173228 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.173230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:53.174050 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:53.174053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.174496 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.174497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:53.174631 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.175287 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:53.175296 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:53.177028 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:53.179279 (+ 2251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:53.179287 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:53.179289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:53.179552 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:53.179860 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:53.179861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:53.179861 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:53.179997 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:53.180290 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:53.180298 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:53.180452 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 10:07:53.180567 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 10:07:53.180769 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":101,"threads_started":1}
I20260808 10:07:53.181856 6598 heartbeater.cc:347] Connected to a master server at 127.2.30.190:40889
I20260808 10:07:53.181989 6598 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:53.182183 6598 heartbeater.cc:511] Master 127.2.30.190:40889 requested a full tablet report, sending...
I20260808 10:07:53.182955 6401 ts_manager.cc:194] Registered new tserver with Master: 289b08fb924f43a9882768df51b3fa1e (127.2.30.129:43515)
I20260808 10:07:53.184038 6401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:57585
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:07:53.206856 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:40889
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:35465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:53.303066 6603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:53.303328 6603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:53.303361 6603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:53.304772 6603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:53.304824 6603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:53.304857 6603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:07:53.306627 6603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:40889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:53.306942 6603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:53.307305 6603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:53.313105 6613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:53.313097 6608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:53.313097 6609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:53.313565 6603 server_base.cc:1034] running on GCE node
I20260808 10:07:53.313756 6603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:53.313970 6603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:53.315236 6603 hybrid_clock.cc:648] HybridClock initialized: now 1786183673315166 us; error 30 us; skew 500 ppm
Aug 08 10:07:53 dist-test-slave-w206 krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183673, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:53.317942 6603 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:07:53.318389 6603 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:53.318475 6603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:53.318631 6603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:53.319854 6603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "373b71d13b5c4c15b1b0d729c246a226"
format_stamp: "Formatted at 2026-08-08 10:07:53 on dist-test-slave-w206"
server_key: "61f41f123708c0d608237f5d380a59ae"
server_key_iv: "2f37ed43a59154ff2edd1565a54ec365"
server_key_version: "encryptionkey@0"
I20260808 10:07:53.320266 6603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "373b71d13b5c4c15b1b0d729c246a226"
format_stamp: "Formatted at 2026-08-08 10:07:53 on dist-test-slave-w206"
server_key: "61f41f123708c0d608237f5d380a59ae"
server_key_iv: "2f37ed43a59154ff2edd1565a54ec365"
server_key_version: "encryptionkey@0"
I20260808 10:07:53.321686 6603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:07:53.322589 6619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:53.322808 6603 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:53.322963 6603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "373b71d13b5c4c15b1b0d729c246a226"
format_stamp: "Formatted at 2026-08-08 10:07:53 on dist-test-slave-w206"
server_key: "61f41f123708c0d608237f5d380a59ae"
server_key_iv: "2f37ed43a59154ff2edd1565a54ec365"
server_key_version: "encryptionkey@0"
I20260808 10:07:53.323112 6603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:53.338387 6603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:53.340839 6603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:53.341030 6603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:53.341310 6603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:53.341714 6603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:53.341776 6603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:53.341821 6603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:53.341846 6603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:53.348536 6603 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:44673
I20260808 10:07:53.348573 6737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:44673 every 8 connection(s)
I20260808 10:07:53.348913 6603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:07:53.353844 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 6603
I20260808 10:07:53.353976 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:07:53.354205 2170 external_mini_cluster.cc:1428] Setting key 4bde35381d22eafc2209557712207384
Aug 08 10:07:53 dist-test-slave-w206 krb5kdc[6354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183673, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:53.360209 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.349747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:43285 (local address 127.2.30.190:40889)
0808 10:07:53.349813 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:53.349815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:53.350131 (+ 316us) server_negotiation.cc:407] Connection header received
0808 10:07:53.350646 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:53.350648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:53.350660 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:53.350730 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:53.351858 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.352411 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.353462 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.353663 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.353726 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.356549 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:53.356550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:53.356551 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:53.356559 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:53.358332 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:53.358889 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:53.358891 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:53.358891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:53.358955 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:53.359319 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:53.359320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:53.359321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:53.359476 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:53.359566 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:53.359967 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 10:07:53.360072 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:07:53.360424 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.349712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:40889 (local address 127.2.30.130:43285)
0808 10:07:53.349936 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:53.349983 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:53.350522 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:53.350818 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:53.350821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:53.351227 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:53.351751 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:53.351756 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.352511 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.352512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:53.353354 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:53.353356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.353761 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.353762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:53.353948 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.354707 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:53.354716 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:53.356434 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:53.358435 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:53.358444 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:53.358447 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:53.358793 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:53.359120 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:53.359121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:53.359121 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:53.359238 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:53.359560 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:53.359567 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:53.359777 (+ 210us) client_negotiation.cc:769] Sending connection context
0808 10:07:53.359942 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 10:07:53.360124 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":120,"threads_started":1}
I20260808 10:07:53.361016 6738 heartbeater.cc:347] Connected to a master server at 127.2.30.190:40889
I20260808 10:07:53.361120 6738 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:53.361308 6738 heartbeater.cc:511] Master 127.2.30.190:40889 requested a full tablet report, sending...
I20260808 10:07:53.361747 6401 ts_manager.cc:194] Registered new tserver with Master: 373b71d13b5c4c15b1b0d729c246a226 (127.2.30.130:44673)
I20260808 10:07:53.362149 6401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:43285
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:07:53.391052 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:40889
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:35465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:53.478817 6743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:53.479147 6743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:53.479182 6743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:53.480590 6743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:53.480646 6743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:53.480676 6743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:07:53.482362 6743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:40889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:53.482652 6743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:53.482914 6743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:53.488832 6749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:53.488847 6753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:53.488847 6750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:53.489130 6743 server_base.cc:1034] running on GCE node
I20260808 10:07:53.489467 6743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:53.489743 6743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:53.490927 6743 hybrid_clock.cc:648] HybridClock initialized: now 1786183673490917 us; error 34 us; skew 500 ppm
Aug 08 10:07:53 dist-test-slave-w206 krb5kdc[6354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183673, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:53.493934 6743 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:07:53.494530 6743 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:53.494622 6743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:53.494809 6743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:53.496157 6743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "96bd3d65a315433ea25dd610cbfb2a28"
format_stamp: "Formatted at 2026-08-08 10:07:53 on dist-test-slave-w206"
server_key: "7ac04564d16213d3423d974f524ca3ca"
server_key_iv: "c7d3db5b870520359b377bda77990cc2"
server_key_version: "encryptionkey@0"
I20260808 10:07:53.496604 6743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "96bd3d65a315433ea25dd610cbfb2a28"
format_stamp: "Formatted at 2026-08-08 10:07:53 on dist-test-slave-w206"
server_key: "7ac04564d16213d3423d974f524ca3ca"
server_key_iv: "c7d3db5b870520359b377bda77990cc2"
server_key_version: "encryptionkey@0"
I20260808 10:07:53.498147 6743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:07:53.499120 6759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:53.499342 6743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:53.499459 6743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "96bd3d65a315433ea25dd610cbfb2a28"
format_stamp: "Formatted at 2026-08-08 10:07:53 on dist-test-slave-w206"
server_key: "7ac04564d16213d3423d974f524ca3ca"
server_key_iv: "c7d3db5b870520359b377bda77990cc2"
server_key_version: "encryptionkey@0"
I20260808 10:07:53.499557 6743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:53.509366 6743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:53.512045 6743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:53.512225 6743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:53.512506 6743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:53.512845 6743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:53.512907 6743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:53.512952 6743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:53.512976 6743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:53.519310 6743 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:45759
I20260808 10:07:53.519389 6874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:45759 every 8 connection(s)
I20260808 10:07:53.519707 6743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
Aug 08 10:07:53 dist-test-slave-w206 krb5kdc[6354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183673, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:53.527597 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 6743
I20260808 10:07:53.527743 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:07:53.528026 2170 external_mini_cluster.cc:1428] Setting key 50ea6f4efb4839f96817bd65786689e0
I20260808 10:07:53.530349 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.520564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:44249 (local address 127.2.30.190:40889)
0808 10:07:53.520657 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:53.520659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:53.520862 (+ 203us) server_negotiation.cc:407] Connection header received
0808 10:07:53.521444 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:53.521446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:53.521461 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:53.521530 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:53.522508 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.523093 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.524207 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.524413 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.524493 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.526935 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:53.526937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:53.526937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:53.526945 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:53.528797 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:53.529307 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:53.529309 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:53.529309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:53.529379 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:53.529629 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:53.529630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:53.529630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:53.529763 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:53.529846 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:53.530163 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 10:07:53.530251 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:07:53.530620 6877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.520528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:40889 (local address 127.2.30.131:44249)
0808 10:07:53.520719 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:53.520760 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:53.521325 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:53.521609 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:53.521613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:53.521891 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:53.522394 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:53.522400 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.523210 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.523211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:53.524066 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:53.524069 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.524506 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.524507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:53.524643 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.525168 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:53.525176 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:53.526815 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:53.528877 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:53.528885 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:53.528888 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:53.529225 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:53.529444 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:53.529445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:53.529445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:53.529551 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:53.529849 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:53.529857 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:53.530014 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 10:07:53.530126 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 10:07:53.530369 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260808 10:07:53.531065 6875 heartbeater.cc:347] Connected to a master server at 127.2.30.190:40889
I20260808 10:07:53.531167 6875 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:53.531297 6875 heartbeater.cc:511] Master 127.2.30.190:40889 requested a full tablet report, sending...
I20260808 10:07:53.531673 6401 ts_manager.cc:194] Registered new tserver with Master: 96bd3d65a315433ea25dd610cbfb2a28 (127.2.30.131:45759)
I20260808 10:07:53.532048 6401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:44249
I20260808 10:07:53.540477 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:07:53.548900 6441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.541992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33510 (local address 127.2.30.190:40889)
0808 10:07:53.542061 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:53.542063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:53.542251 (+ 188us) server_negotiation.cc:407] Connection header received
0808 10:07:53.542299 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:53.542299 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:53.542309 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:53.542360 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:53.543083 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.543599 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.544330 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.544510 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.544562 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.545819 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:53.545821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:53.545821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:53.545827 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:53.547455 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:53.547880 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:53.547881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:53.547881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:53.547939 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:53.548180 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:53.548181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:53.548181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:53.548360 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:53.548445 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:53.548632 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 10:07:53.548707 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 10:07:53.550926 6401 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:07:53.552660 6401 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:07:53.559432 6887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.555312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:44673 (local address 127.0.0.1:51508)
0808 10:07:53.555496 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:53.555540 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:53.555609 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:53.556040 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:53.556041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:53.556043 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:53.556103 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:53.556105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.557024 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.557026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:53.558086 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:53.558089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.559120 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.559121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:53.559206 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.559221 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 10:07:53.559292 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 10:07:53.559345 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":108,"threads_started":1}
I20260808 10:07:53.559456 6891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.555533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51508 (local address 127.2.30.130:44673)
0808 10:07:53.555856 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:53.555860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:53.555869 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:07:53.555906 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:53.555908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:53.555922 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:53.556023 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:53.556167 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.556921 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.558188 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.559039 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.559103 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.559167 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:53.559325 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 10:07:53.559368 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":89,"threads_started":1}
I20260808 10:07:53.560022 6886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.555312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:43515 (local address 127.0.0.1:47664)
0808 10:07:53.555708 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:53.555718 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:53.555757 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:53.555958 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:53.555961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:53.555964 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:53.556046 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:53.556048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.557632 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.557633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:53.558711 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:53.558714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.559741 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.559741 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:53.559813 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.559817 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:07:53.559868 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 10:07:53.559951 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":25,"thread_start_us":52,"threads_started":1}
I20260808 10:07:53.560206 6892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.555651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41638 (local address 127.2.30.131:45759)
0808 10:07:53.555847 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:53.555852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:53.556241 (+ 389us) server_negotiation.cc:407] Connection header received
0808 10:07:53.556302 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:53.556304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:53.556321 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:53.556389 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:53.556591 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.557654 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.558874 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.559680 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.559745 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.559795 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:53.560071 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 10:07:53.560118 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":87,"threads_started":1}
I20260808 10:07:53.560333 6889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.555526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47664 (local address 127.2.30.129:43515)
0808 10:07:53.555767 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:53.555772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:53.555783 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:07:53.555823 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:53.555825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:53.555840 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:53.555943 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:53.556376 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.557517 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.558815 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.559665 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.559737 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.559788 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:53.560209 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 10:07:53.560252 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":94,"threads_started":1}
I20260808 10:07:53.560371 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.555384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:45759 (local address 127.0.0.1:41638)
0808 10:07:53.556173 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:53.556182 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:53.556235 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:53.556478 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:53.556479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:53.556481 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:53.556514 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:53.556515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.557767 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.557768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:53.558787 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:53.558790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.559833 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.559834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:53.559921 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.559926 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:07:53.559982 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 10:07:53.560024 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":771,"mutex_wait_us":2,"thread_start_us":113,"threads_started":1}
I20260808 10:07:53.560668 6668 tablet_service.cc:1467] Processing CreateTablet for tablet 4804081400104057b0f2623e33c0183a (DEFAULT_TABLE table=test-table [id=6e8eb6a915f84c6da438f9504f5c83c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:53.561024 6668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4804081400104057b0f2623e33c0183a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:53.561005 6531 tablet_service.cc:1467] Processing CreateTablet for tablet 4804081400104057b0f2623e33c0183a (DEFAULT_TABLE table=test-table [id=6e8eb6a915f84c6da438f9504f5c83c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:53.561036 6807 tablet_service.cc:1467] Processing CreateTablet for tablet 4804081400104057b0f2623e33c0183a (DEFAULT_TABLE table=test-table [id=6e8eb6a915f84c6da438f9504f5c83c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:53.561352 6807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4804081400104057b0f2623e33c0183a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:53.561352 6531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4804081400104057b0f2623e33c0183a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:53.563795 6893 tablet_bootstrap.cc:492] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226: Bootstrap starting.
I20260808 10:07:53.564479 6893 tablet_bootstrap.cc:654] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:53.564917 6895 tablet_bootstrap.cc:492] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28: Bootstrap starting.
I20260808 10:07:53.565109 6893 log.cc:824] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:53.565423 6895 tablet_bootstrap.cc:654] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:53.565927 6893 tablet_bootstrap.cc:492] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226: No bootstrap required, opened a new log
I20260808 10:07:53.565929 6895 log.cc:824] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:53.565987 6893 ts_tablet_manager.cc:1397] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:07:53.566260 6894 tablet_bootstrap.cc:492] T 4804081400104057b0f2623e33c0183a P 289b08fb924f43a9882768df51b3fa1e: Bootstrap starting.
I20260808 10:07:53.566659 6895 tablet_bootstrap.cc:492] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28: No bootstrap required, opened a new log
I20260808 10:07:53.566746 6895 ts_tablet_manager.cc:1397] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:07:53.567605 6894 tablet_bootstrap.cc:654] T 4804081400104057b0f2623e33c0183a P 289b08fb924f43a9882768df51b3fa1e: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:53.567956 6893 raft_consensus.cc:348] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373b71d13b5c4c15b1b0d729c246a226" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 44673 } } peers { permanent_uuid: "289b08fb924f43a9882768df51b3fa1e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43515 } } peers { permanent_uuid: "96bd3d65a315433ea25dd610cbfb2a28" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45759 } }
I20260808 10:07:53.568127 6893 raft_consensus.cc:374] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:53.568171 6893 raft_consensus.cc:729] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373b71d13b5c4c15b1b0d729c246a226, State: Initialized, Role: FOLLOWER
I20260808 10:07:53.568193 6894 log.cc:824] T 4804081400104057b0f2623e33c0183a P 289b08fb924f43a9882768df51b3fa1e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:53.568327 6893 consensus_queue.cc:260] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373b71d13b5c4c15b1b0d729c246a226" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 44673 } } peers { permanent_uuid: "289b08fb924f43a9882768df51b3fa1e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43515 } } peers { permanent_uuid: "96bd3d65a315433ea25dd610cbfb2a28" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45759 } }
I20260808 10:07:53.568671 6893 ts_tablet_manager.cc:1428] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:07:53.568828 6738 heartbeater.cc:503] Master 127.2.30.190:40889 was elected leader, sending a full tablet report...
I20260808 10:07:53.569085 6894 tablet_bootstrap.cc:492] T 4804081400104057b0f2623e33c0183a P 289b08fb924f43a9882768df51b3fa1e: No bootstrap required, opened a new log
I20260808 10:07:53.569005 6895 raft_consensus.cc:348] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373b71d13b5c4c15b1b0d729c246a226" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 44673 } } peers { permanent_uuid: "289b08fb924f43a9882768df51b3fa1e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43515 } } peers { permanent_uuid: "96bd3d65a315433ea25dd610cbfb2a28" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45759 } }
I20260808 10:07:53.569156 6894 ts_tablet_manager.cc:1397] T 4804081400104057b0f2623e33c0183a P 289b08fb924f43a9882768df51b3fa1e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:07:53.569185 6895 raft_consensus.cc:374] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:53.569248 6895 raft_consensus.cc:729] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96bd3d65a315433ea25dd610cbfb2a28, State: Initialized, Role: FOLLOWER
I20260808 10:07:53.569376 6895 consensus_queue.cc:260] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373b71d13b5c4c15b1b0d729c246a226" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 44673 } } peers { permanent_uuid: "289b08fb924f43a9882768df51b3fa1e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43515 } } peers { permanent_uuid: "96bd3d65a315433ea25dd610cbfb2a28" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45759 } }
I20260808 10:07:53.569653 6895 ts_tablet_manager.cc:1428] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:07:53.569667 6875 heartbeater.cc:503] Master 127.2.30.190:40889 was elected leader, sending a full tablet report...
I20260808 10:07:53.571324 6894 raft_consensus.cc:348] T 4804081400104057b0f2623e33c0183a P 289b08fb924f43a9882768df51b3fa1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373b71d13b5c4c15b1b0d729c246a226" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 44673 } } peers { permanent_uuid: "289b08fb924f43a9882768df51b3fa1e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43515 } } peers { permanent_uuid: "96bd3d65a315433ea25dd610cbfb2a28" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45759 } }
I20260808 10:07:53.571453 6894 raft_consensus.cc:374] T 4804081400104057b0f2623e33c0183a P 289b08fb924f43a9882768df51b3fa1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:53.571475 6894 raft_consensus.cc:729] T 4804081400104057b0f2623e33c0183a P 289b08fb924f43a9882768df51b3fa1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 289b08fb924f43a9882768df51b3fa1e, State: Initialized, Role: FOLLOWER
I20260808 10:07:53.571590 6894 consensus_queue.cc:260] T 4804081400104057b0f2623e33c0183a P 289b08fb924f43a9882768df51b3fa1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373b71d13b5c4c15b1b0d729c246a226" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 44673 } } peers { permanent_uuid: "289b08fb924f43a9882768df51b3fa1e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43515 } } peers { permanent_uuid: "96bd3d65a315433ea25dd610cbfb2a28" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45759 } }
I20260808 10:07:53.571839 6894 ts_tablet_manager.cc:1428] T 4804081400104057b0f2623e33c0183a P 289b08fb924f43a9882768df51b3fa1e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 10:07:53.571915 6598 heartbeater.cc:503] Master 127.2.30.190:40889 was elected leader, sending a full tablet report...
W20260808 10:07:53.599514 6739 tablet.cc:2369] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:07:53.670516 6599 tablet.cc:2369] T 4804081400104057b0f2623e33c0183a P 289b08fb924f43a9882768df51b3fa1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:07:53.734170 6899 raft_consensus.cc:482] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:07:53.734386 6899 raft_consensus.cc:504] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373b71d13b5c4c15b1b0d729c246a226" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 44673 } } peers { permanent_uuid: "289b08fb924f43a9882768df51b3fa1e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43515 } } peers { permanent_uuid: "96bd3d65a315433ea25dd610cbfb2a28" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45759 } }
I20260808 10:07:53.734809 6899 leader_election.cc:288] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 289b08fb924f43a9882768df51b3fa1e (127.2.30.129:43515), 96bd3d65a315433ea25dd610cbfb2a28 (127.2.30.131:45759)
I20260808 10:07:53.738025 6889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.735183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:58549 (local address 127.2.30.129:43515)
0808 10:07:53.735230 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:53.735232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:53.735472 (+ 240us) server_negotiation.cc:407] Connection header received
0808 10:07:53.735555 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:53.735556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:53.735564 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:53.735618 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:53.735794 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.736259 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.737254 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.737633 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.737727 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.737737 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:53.737900 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 10:07:53.737940 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:07:53.738050 6902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.735148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:43515 (local address 127.2.30.130:58549)
0808 10:07:53.735379 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:53.735388 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:53.735485 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:53.735623 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:53.735623 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:53.735625 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:53.735711 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:53.735713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.736383 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.736384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:53.737139 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:53.737141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.737737 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.737738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:53.737819 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.737823 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:07:53.737893 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 10:07:53.737940 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":1,"thread_start_us":120,"threads_started":1}
I20260808 10:07:53.738045 6892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.735280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:59395 (local address 127.2.30.131:45759)
0808 10:07:53.735351 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:53.735353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:53.735361 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:07:53.735386 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:53.735387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:53.735398 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:53.735472 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:07:53.735651 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.736130 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.737241 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.737639 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.737738 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.737747 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:53.737914 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 10:07:53.737958 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 10:07:53.738070 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.735125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:45759 (local address 127.2.30.130:59395)
0808 10:07:53.735189 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:53.735202 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:53.735256 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:53.735492 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:53.735493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:53.735495 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:07:53.735569 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:53.735571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.736320 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.736321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:53.737137 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:53.737139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.737747 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.737748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:53.737830 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.737833 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:07:53.737896 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 10:07:53.737941 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 10:07:53.738451 6552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4804081400104057b0f2623e33c0183a" candidate_uuid: "373b71d13b5c4c15b1b0d729c246a226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "289b08fb924f43a9882768df51b3fa1e" is_pre_election: true
I20260808 10:07:53.738451 6828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4804081400104057b0f2623e33c0183a" candidate_uuid: "373b71d13b5c4c15b1b0d729c246a226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96bd3d65a315433ea25dd610cbfb2a28" is_pre_election: true
I20260808 10:07:53.738641 6552 raft_consensus.cc:2454] T 4804081400104057b0f2623e33c0183a P 289b08fb924f43a9882768df51b3fa1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373b71d13b5c4c15b1b0d729c246a226 in term 0.
I20260808 10:07:53.738632 6828 raft_consensus.cc:2454] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 373b71d13b5c4c15b1b0d729c246a226 in term 0.
I20260808 10:07:53.738956 6622 leader_election.cc:302] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 289b08fb924f43a9882768df51b3fa1e, 373b71d13b5c4c15b1b0d729c246a226; no voters:
I20260808 10:07:53.739152 6899 raft_consensus.cc:2781] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:07:53.739308 6899 raft_consensus.cc:482] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:07:53.739351 6899 raft_consensus.cc:3037] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:53.740115 6899 raft_consensus.cc:504] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373b71d13b5c4c15b1b0d729c246a226" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 44673 } } peers { permanent_uuid: "289b08fb924f43a9882768df51b3fa1e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43515 } } peers { permanent_uuid: "96bd3d65a315433ea25dd610cbfb2a28" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45759 } }
I20260808 10:07:53.740293 6899 leader_election.cc:288] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [CANDIDATE]: Term 1 election: Requested vote from peers 289b08fb924f43a9882768df51b3fa1e (127.2.30.129:43515), 96bd3d65a315433ea25dd610cbfb2a28 (127.2.30.131:45759)
I20260808 10:07:53.740483 6552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4804081400104057b0f2623e33c0183a" candidate_uuid: "373b71d13b5c4c15b1b0d729c246a226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "289b08fb924f43a9882768df51b3fa1e"
I20260808 10:07:53.740597 6552 raft_consensus.cc:3037] T 4804081400104057b0f2623e33c0183a P 289b08fb924f43a9882768df51b3fa1e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:53.741654 6552 raft_consensus.cc:2454] T 4804081400104057b0f2623e33c0183a P 289b08fb924f43a9882768df51b3fa1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373b71d13b5c4c15b1b0d729c246a226 in term 1.
I20260808 10:07:53.741782 6828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4804081400104057b0f2623e33c0183a" candidate_uuid: "373b71d13b5c4c15b1b0d729c246a226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96bd3d65a315433ea25dd610cbfb2a28"
I20260808 10:07:53.741842 6622 leader_election.cc:302] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 289b08fb924f43a9882768df51b3fa1e, 373b71d13b5c4c15b1b0d729c246a226; no voters:
I20260808 10:07:53.741880 6828 raft_consensus.cc:3037] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:53.741948 6899 raft_consensus.cc:2781] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:53.742138 6899 raft_consensus.cc:686] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [term 1 LEADER]: Becoming Leader. State: Replica: 373b71d13b5c4c15b1b0d729c246a226, State: Running, Role: LEADER
I20260808 10:07:53.742267 6899 consensus_queue.cc:237] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373b71d13b5c4c15b1b0d729c246a226" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 44673 } } peers { permanent_uuid: "289b08fb924f43a9882768df51b3fa1e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43515 } } peers { permanent_uuid: "96bd3d65a315433ea25dd610cbfb2a28" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45759 } }
I20260808 10:07:53.742780 6828 raft_consensus.cc:2454] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 373b71d13b5c4c15b1b0d729c246a226 in term 1.
I20260808 10:07:53.743198 6400 catalog_manager.cc:5495] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 reported cstate change: term changed from 0 to 1, leader changed from <none> to 373b71d13b5c4c15b1b0d729c246a226 (127.2.30.130). New cstate: current_term: 1 leader_uuid: "373b71d13b5c4c15b1b0d729c246a226" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "373b71d13b5c4c15b1b0d729c246a226" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 44673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "289b08fb924f43a9882768df51b3fa1e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96bd3d65a315433ea25dd610cbfb2a28" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45759 } health_report { overall_health: UNKNOWN } } }
W20260808 10:07:53.770397 6876 tablet.cc:2369] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:07:53.792274 6891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.789663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51514 (local address 127.2.30.130:44673)
0808 10:07:53.789728 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:53.789730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:53.789744 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:07:53.789785 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:53.789786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:53.789809 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:53.789884 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:07:53.790111 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.790600 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.791441 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:53.791651 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:53.791745 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:53.791880 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:07:53.791991 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:07:53.792056 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:53.792093 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 10:07:53.792115 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:07:53.794497 6552 raft_consensus.cc:1261] T 4804081400104057b0f2623e33c0183a P 289b08fb924f43a9882768df51b3fa1e [term 1 FOLLOWER]: Refusing update from remote peer 373b71d13b5c4c15b1b0d729c246a226: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:53.794498 6828 raft_consensus.cc:1261] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28 [term 1 FOLLOWER]: Refusing update from remote peer 373b71d13b5c4c15b1b0d729c246a226: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:53.794915 6899 consensus_queue.cc:1035] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96bd3d65a315433ea25dd610cbfb2a28" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:53.795085 6903 consensus_queue.cc:1035] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [LEADER]: Connected to new peer: Peer: permanent_uuid: "289b08fb924f43a9882768df51b3fa1e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:53.797904 6907 mvcc.cc:205] Tried to move back new op lower bound from 7316208327858724864 to 7316208327649046528. Current Snapshot: MvccSnapshot[applied={T|T < 7316208327858724864}]
I20260808 10:07:53.798645 6908 mvcc.cc:205] Tried to move back new op lower bound from 7316208327858724864 to 7316208327649046528. Current Snapshot: MvccSnapshot[applied={T|T < 7316208327858724864}]
I20260808 10:07:53.799160 6906 mvcc.cc:205] Tried to move back new op lower bound from 7316208327858724864 to 7316208327649046528. Current Snapshot: MvccSnapshot[applied={T|T < 7316208327858724864 or (T in {7316208327858724864})}]
I20260808 10:07:53.800216 6401 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:07:53.800328 6401 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:07:53.801141 6401 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f27a499e22242519b1d492959e6efe2: Sending DeleteTablet for 3 replicas of tablet 4804081400104057b0f2623e33c0183a
I20260808 10:07:53.801441 6668 tablet_service.cc:1514] Processing DeleteTablet for tablet 4804081400104057b0f2623e33c0183a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:53 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:51508
I20260808 10:07:53.801519 6807 tablet_service.cc:1514] Processing DeleteTablet for tablet 4804081400104057b0f2623e33c0183a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:53 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:41638
I20260808 10:07:53.801719 6916 tablet_replica.cc:330] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226: stopping tablet replica
I20260808 10:07:53.801739 6531 tablet_service.cc:1514] Processing DeleteTablet for tablet 4804081400104057b0f2623e33c0183a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:53 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:47664
I20260808 10:07:53.801822 6916 raft_consensus.cc:2229] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:07:53.801915 6916 raft_consensus.cc:2258] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:53.801956 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 6465
I20260808 10:07:53.802373 6916 ts_tablet_manager.cc:1905] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:53.802959 6917 tablet_replica.cc:330] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28: stopping tablet replica
I20260808 10:07:53.803174 6917 raft_consensus.cc:2229] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:07:53.803330 6917 raft_consensus.cc:2258] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:53.803814 6917 ts_tablet_manager.cc:1905] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:53.804665 6916 ts_tablet_manager.cc:1918] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:07:53.804745 6916 log.cc:1196] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/4804081400104057b0f2623e33c0183a
I20260808 10:07:53.805044 6916 ts_tablet_manager.cc:1939] T 4804081400104057b0f2623e33c0183a P 373b71d13b5c4c15b1b0d729c246a226: Deleting consensus metadata
I20260808 10:07:53.805660 6385 catalog_manager.cc:4841] TS 373b71d13b5c4c15b1b0d729c246a226 (127.2.30.130:44673): tablet 4804081400104057b0f2623e33c0183a (table test-table [id=6e8eb6a915f84c6da438f9504f5c83c1]) successfully deleted
I20260808 10:07:53.806329 6917 ts_tablet_manager.cc:1918] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:07:53.806425 6917 log.cc:1196] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/4804081400104057b0f2623e33c0183a
I20260808 10:07:53.806751 6917 ts_tablet_manager.cc:1939] T 4804081400104057b0f2623e33c0183a P 96bd3d65a315433ea25dd610cbfb2a28: Deleting consensus metadata
I20260808 10:07:53.807411 6386 catalog_manager.cc:4841] TS 96bd3d65a315433ea25dd610cbfb2a28 (127.2.30.131:45759): tablet 4804081400104057b0f2623e33c0183a (table test-table [id=6e8eb6a915f84c6da438f9504f5c83c1]) successfully deleted
W20260808 10:07:53.810689 6387 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:43515 (error 108)
I20260808 10:07:53.810966 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 6603
I20260808 10:07:53.813257 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.813103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:43515 (local address 127.0.0.1:47672)
0808 10:07:53.813170 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:53.813218 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:43515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 10:07:53.813498 6387 catalog_manager.cc:4584] TS 289b08fb924f43a9882768df51b3fa1e (127.2.30.129:43515): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4804081400104057b0f2623e33c0183a: Network error: Client connection negotiation failed: client connection to 127.2.30.129:43515: connect: Connection refused (error 111)
I20260808 10:07:53.819216 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 6743
I20260808 10:07:53.825088 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 6368
2026-08-08T10:07:53Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2256 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T10:07:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:53Z Disabled control of system clock
I20260808 10:07:53.856096 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:41797
--webserver_interface=127.2.30.190
--webserver_port=0
--builtin_ntp_servers=127.2.30.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:41797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 10:07:53.941023 6925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:53.941244 6925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:53.941267 6925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:53.942417 6925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:53.942463 6925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:53.942476 6925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:53.942489 6925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:53.942504 6925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:53.943954 6925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:41797
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:41797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:53.944172 6925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:53.944446 6925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:07:53.949808 6925 server_base.cc:1034] running on GCE node
W20260808 10:07:53.949816 6930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:53.949759 6931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:53.949760 6933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:53.950438 6925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:53.950767 6925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:53.951967 6925 hybrid_clock.cc:648] HybridClock initialized: now 1786183673951940 us; error 64 us; skew 500 ppm
I20260808 10:07:53.953460 6925 webserver.cc:460] Webserver started at http://127.2.30.190:44263/ using document root <none> and password file <none>
I20260808 10:07:53.953718 6925 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:53.953788 6925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:53.953950 6925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:53.955233 6925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "ad4fdfc15b3a499c86b4fed5d0b9e553"
format_stamp: "Formatted at 2026-08-08 10:07:53 on dist-test-slave-w206"
server_key: "f4e44bb1859a9441e4f97b58a1e25728"
server_key_iv: "9c794c3c6d8ddef1509281afb9702113"
server_key_version: "encryptionkey@0"
I20260808 10:07:53.955628 6925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "ad4fdfc15b3a499c86b4fed5d0b9e553"
format_stamp: "Formatted at 2026-08-08 10:07:53 on dist-test-slave-w206"
server_key: "f4e44bb1859a9441e4f97b58a1e25728"
server_key_iv: "9c794c3c6d8ddef1509281afb9702113"
server_key_version: "encryptionkey@0"
I20260808 10:07:53.957070 6925 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:07:53.957949 6939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:53.958148 6925 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:53.958261 6925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "ad4fdfc15b3a499c86b4fed5d0b9e553"
format_stamp: "Formatted at 2026-08-08 10:07:53 on dist-test-slave-w206"
server_key: "f4e44bb1859a9441e4f97b58a1e25728"
server_key_iv: "9c794c3c6d8ddef1509281afb9702113"
server_key_version: "encryptionkey@0"
I20260808 10:07:53.958384 6925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:53.962047 6925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:53.962357 6925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:53.962507 6925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:53.966747 6925 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:41797
I20260808 10:07:53.966890 6993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:41797 every 8 connection(s)
I20260808 10:07:53.967221 6925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:53.967940 6994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:53.970933 6994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553: Bootstrap starting.
I20260808 10:07:53.971335 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 6925
I20260808 10:07:53.971460 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:07:53.971683 2170 external_mini_cluster.cc:1428] Setting key dece619bafb0be6bced351728bc87d02
I20260808 10:07:53.971746 6994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:53.972105 6994 log.cc:824] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:53.973034 6994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553: No bootstrap required, opened a new log
I20260808 10:07:53.974676 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:53.972314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37944 (local address 127.2.30.190:41797)
0808 10:07:53.972571 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:53.972579 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:53.972604 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:07:53.973215 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:53.973225 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:53.973275 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:53.973449 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:53.974183 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:53.974187 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:53.974187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:53.974193 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:53.974249 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:53.974263 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:53.974371 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:53.974405 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 10:07:53.974466 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":119,"threads_started":1}
I20260808 10:07:53.975098 6994 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4fdfc15b3a499c86b4fed5d0b9e553" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41797 } }
I20260808 10:07:53.975245 6994 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:53.975267 6994 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad4fdfc15b3a499c86b4fed5d0b9e553, State: Initialized, Role: FOLLOWER
I20260808 10:07:53.975401 6994 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4fdfc15b3a499c86b4fed5d0b9e553" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41797 } }
I20260808 10:07:53.975669 6994 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:53.975714 6994 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:53.975800 6994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:53.976816 6994 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4fdfc15b3a499c86b4fed5d0b9e553" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41797 } }
I20260808 10:07:53.976919 6994 leader_election.cc:302] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad4fdfc15b3a499c86b4fed5d0b9e553; no voters:
I20260808 10:07:53.977241 6994 leader_election.cc:288] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:53.977315 6999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:53.977491 6999 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [term 1 LEADER]: Becoming Leader. State: Replica: ad4fdfc15b3a499c86b4fed5d0b9e553, State: Running, Role: LEADER
I20260808 10:07:53.977608 6994 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:53.977594 6999 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4fdfc15b3a499c86b4fed5d0b9e553" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41797 } }
I20260808 10:07:53.978096 7000 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad4fdfc15b3a499c86b4fed5d0b9e553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4fdfc15b3a499c86b4fed5d0b9e553" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41797 } } }
I20260808 10:07:53.978075 7001 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad4fdfc15b3a499c86b4fed5d0b9e553. Latest consensus state: current_term: 1 leader_uuid: "ad4fdfc15b3a499c86b4fed5d0b9e553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4fdfc15b3a499c86b4fed5d0b9e553" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41797 } } }
I20260808 10:07:53.978230 7000 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:53.978250 7001 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:53.978766 7007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:53.979344 7007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:53.981038 7007 catalog_manager.cc:1347] Generated new cluster ID: 78faffc3ffda41daaa56d9657bf3cb3b
I20260808 10:07:53.981123 7007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:53.999358 7007 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:53.999595 7007 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:54.002589 7007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553: Generated new TSK 0
I20260808 10:07:54.002774 7007 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:54.004369 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41797
--builtin_ntp_servers=127.2.30.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 10:07:54.095520 7019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:54.095739 7019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:54.095763 7019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:54.096995 7019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:54.097038 7019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:54.097064 7019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:07:54.098464 7019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:54.098742 7019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:54.099112 7019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:54.104480 7027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:54.104466 7024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:54.104471 7025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:54.105057 7019 server_base.cc:1034] running on GCE node
I20260808 10:07:54.105342 7019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:54.105679 7019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:54.106890 7019 hybrid_clock.cc:648] HybridClock initialized: now 1786183674106869 us; error 36 us; skew 500 ppm
I20260808 10:07:54.108608 7019 webserver.cc:460] Webserver started at http://127.2.30.129:41575/ using document root <none> and password file <none>
I20260808 10:07:54.108917 7019 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:54.108991 7019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:54.109160 7019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:54.110706 7019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "4204a91d17d54c6b8060f7da6c740da8"
format_stamp: "Formatted at 2026-08-08 10:07:54 on dist-test-slave-w206"
server_key: "72ad2b68e559eeb960f0e231be054972"
server_key_iv: "70bc5736640e59c3f5119de76e0102a8"
server_key_version: "encryptionkey@0"
I20260808 10:07:54.111223 7019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "4204a91d17d54c6b8060f7da6c740da8"
format_stamp: "Formatted at 2026-08-08 10:07:54 on dist-test-slave-w206"
server_key: "72ad2b68e559eeb960f0e231be054972"
server_key_iv: "70bc5736640e59c3f5119de76e0102a8"
server_key_version: "encryptionkey@0"
I20260808 10:07:54.113190 7019 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260808 10:07:54.114207 7033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:54.114470 7019 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:54.114591 7019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "4204a91d17d54c6b8060f7da6c740da8"
format_stamp: "Formatted at 2026-08-08 10:07:54 on dist-test-slave-w206"
server_key: "72ad2b68e559eeb960f0e231be054972"
server_key_iv: "70bc5736640e59c3f5119de76e0102a8"
server_key_version: "encryptionkey@0"
I20260808 10:07:54.114677 7019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:54.118183 7019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:54.118525 7019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:54.118688 7019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:54.118917 7019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:54.119305 7019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:54.119372 7019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:54.119411 7019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:54.119439 7019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:54.125186 7019 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:43711
I20260808 10:07:54.125273 7150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:43711 every 8 connection(s)
I20260808 10:07:54.125633 7019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:07:54.129087 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.126544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:50159 (local address 127.2.30.190:41797)
0808 10:07:54.126617 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:54.126619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:54.126883 (+ 264us) server_negotiation.cc:407] Connection header received
0808 10:07:54.127425 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:54.127426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:54.127433 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:54.127520 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:54.128764 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:54.128766 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:54.128767 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:54.128774 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:54.128796 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:54.128803 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:54.128894 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:54.128987 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 10:07:54.129005 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:07:54.129233 7153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.126527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41797 (local address 127.2.30.129:50159)
0808 10:07:54.126737 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:54.126786 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:54.127327 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:54.127588 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:54.127591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:54.127600 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:54.128593 (+ 993us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:07:54.128593 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:54.128622 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:07:54.128652 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:54.128898 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:54.128904 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:54.128906 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:07:54.128968 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 10:07:54.129019 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":112,"threads_started":1}
I20260808 10:07:54.129788 7151 heartbeater.cc:347] Connected to a master server at 127.2.30.190:41797
I20260808 10:07:54.129892 7151 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:54.130052 7151 heartbeater.cc:511] Master 127.2.30.190:41797 requested a full tablet report, sending...
I20260808 10:07:54.130695 6956 ts_manager.cc:194] Registered new tserver with Master: 4204a91d17d54c6b8060f7da6c740da8 (127.2.30.129:43711)
I20260808 10:07:54.131124 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 7019
I20260808 10:07:54.131225 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:07:54.131413 2170 external_mini_cluster.cc:1428] Setting key 58870142cf73c4934adac81b942f6358
I20260808 10:07:54.132404 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41797
--builtin_ntp_servers=127.2.30.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 10:07:54.218492 7154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:54.218801 7154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:54.218853 7154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:54.220269 7154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:54.220348 7154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:54.220394 7154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:07:54.221894 7154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:54.222195 7154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:54.222532 7154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:54.228117 7159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:54.228117 7162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:54.228150 7160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:54.228761 7154 server_base.cc:1034] running on GCE node
I20260808 10:07:54.229028 7154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:54.229386 7154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:54.230571 7154 hybrid_clock.cc:648] HybridClock initialized: now 1786183674230528 us; error 59 us; skew 500 ppm
I20260808 10:07:54.232273 7154 webserver.cc:460] Webserver started at http://127.2.30.130:45589/ using document root <none> and password file <none>
I20260808 10:07:54.232558 7154 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:54.232631 7154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:54.232786 7154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:54.234587 7154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "36f2d19ec8e44ea8b5d2305809b0afb7"
format_stamp: "Formatted at 2026-08-08 10:07:54 on dist-test-slave-w206"
server_key: "4fc640cd5fc6ea098b059bda7796adae"
server_key_iv: "dea3bb85ffb8016eff4415863c6efff8"
server_key_version: "encryptionkey@0"
I20260808 10:07:54.235208 7154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "36f2d19ec8e44ea8b5d2305809b0afb7"
format_stamp: "Formatted at 2026-08-08 10:07:54 on dist-test-slave-w206"
server_key: "4fc640cd5fc6ea098b059bda7796adae"
server_key_iv: "dea3bb85ffb8016eff4415863c6efff8"
server_key_version: "encryptionkey@0"
I20260808 10:07:54.238209 7154 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260808 10:07:54.239708 7168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:54.239879 7154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:54.239998 7154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "36f2d19ec8e44ea8b5d2305809b0afb7"
format_stamp: "Formatted at 2026-08-08 10:07:54 on dist-test-slave-w206"
server_key: "4fc640cd5fc6ea098b059bda7796adae"
server_key_iv: "dea3bb85ffb8016eff4415863c6efff8"
server_key_version: "encryptionkey@0"
I20260808 10:07:54.240103 7154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:54.244812 7154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:54.245182 7154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:54.245334 7154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:54.245599 7154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:54.245971 7154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:54.246033 7154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:54.246078 7154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:54.246104 7154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:54.252202 7154 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:38955
I20260808 10:07:54.252240 7284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:38955 every 8 connection(s)
I20260808 10:07:54.252609 7154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:07:54.256086 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.253489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:49661 (local address 127.2.30.190:41797)
0808 10:07:54.253569 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:54.253571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:54.253836 (+ 265us) server_negotiation.cc:407] Connection header received
0808 10:07:54.254338 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:54.254339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:54.254345 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:54.254417 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:54.255801 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:54.255803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:54.255803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:54.255808 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:54.255825 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:54.255829 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:54.255893 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:54.256011 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 10:07:54.256027 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:07:54.256325 7287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.253451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41797 (local address 127.2.30.130:49661)
0808 10:07:54.253666 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:54.253708 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:54.254195 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:54.254524 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:54.254529 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:54.254537 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:54.255628 (+ 1091us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:07:54.255629 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:54.255660 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:07:54.255690 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:54.255905 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:54.255909 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:54.255912 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:07:54.255980 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 10:07:54.256033 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":115,"threads_started":1}
I20260808 10:07:54.256853 7285 heartbeater.cc:347] Connected to a master server at 127.2.30.190:41797
I20260808 10:07:54.256951 7285 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:54.257071 7285 heartbeater.cc:511] Master 127.2.30.190:41797 requested a full tablet report, sending...
I20260808 10:07:54.257507 6956 ts_manager.cc:194] Registered new tserver with Master: 36f2d19ec8e44ea8b5d2305809b0afb7 (127.2.30.130:38955)
I20260808 10:07:54.258680 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 7154
I20260808 10:07:54.258839 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:07:54.259112 2170 external_mini_cluster.cc:1428] Setting key 65ec6ae775ecc023a12fb1f05dbc8784
I20260808 10:07:54.260360 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41797
--builtin_ntp_servers=127.2.30.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 10:07:54.347993 7288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:54.348227 7288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:54.348253 7288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:54.349406 7288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:54.349442 7288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:54.349468 7288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:07:54.350736 7288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:54.350939 7288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:54.351253 7288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:54.356743 7294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:54.356906 7288 server_base.cc:1034] running on GCE node
W20260808 10:07:54.356729 7293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:54.356731 7296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:54.357286 7288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:54.357527 7288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:54.358731 7288 hybrid_clock.cc:648] HybridClock initialized: now 1786183674358719 us; error 34 us; skew 500 ppm
I20260808 10:07:54.360123 7288 webserver.cc:460] Webserver started at http://127.2.30.131:37243/ using document root <none> and password file <none>
I20260808 10:07:54.360433 7288 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:54.360487 7288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:54.360632 7288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:54.361972 7288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "38a14dcd7cd84b6eba75c1dd9a4546b7"
format_stamp: "Formatted at 2026-08-08 10:07:54 on dist-test-slave-w206"
server_key: "bc745fbafaeff85757b5bbe15e9e86f2"
server_key_iv: "3278909f03c4a7c55bd306426591ba5b"
server_key_version: "encryptionkey@0"
I20260808 10:07:54.362450 7288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "38a14dcd7cd84b6eba75c1dd9a4546b7"
format_stamp: "Formatted at 2026-08-08 10:07:54 on dist-test-slave-w206"
server_key: "bc745fbafaeff85757b5bbe15e9e86f2"
server_key_iv: "3278909f03c4a7c55bd306426591ba5b"
server_key_version: "encryptionkey@0"
I20260808 10:07:54.364099 7288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:07:54.364996 7302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:54.365214 7288 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:54.365319 7288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "38a14dcd7cd84b6eba75c1dd9a4546b7"
format_stamp: "Formatted at 2026-08-08 10:07:54 on dist-test-slave-w206"
server_key: "bc745fbafaeff85757b5bbe15e9e86f2"
server_key_iv: "3278909f03c4a7c55bd306426591ba5b"
server_key_version: "encryptionkey@0"
I20260808 10:07:54.365419 7288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:54.369172 7288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:54.369498 7288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:54.369644 7288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:54.369863 7288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:54.370185 7288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:54.370296 7288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:54.370339 7288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:54.370366 7288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:54.376809 7288 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:40515
I20260808 10:07:54.376847 7418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:40515 every 8 connection(s)
I20260808 10:07:54.377221 7288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:07:54.380997 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.378135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:55013 (local address 127.2.30.190:41797)
0808 10:07:54.378219 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:54.378221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:54.378551 (+ 330us) server_negotiation.cc:407] Connection header received
0808 10:07:54.379228 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:54.379230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:54.379238 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:54.379288 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:54.380676 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:54.380677 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:54.380678 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:54.380682 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:54.380699 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:54.380703 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:54.380770 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:54.380867 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 10:07:54.380909 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 10:07:54.381124 7421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.378127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41797 (local address 127.2.30.131:55013)
0808 10:07:54.378375 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:54.378454 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:54.379114 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:54.379369 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:54.379373 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:54.379384 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:54.380420 (+ 1036us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:07:54.380421 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:54.380452 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:07:54.380482 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:54.380790 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:54.380795 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:54.380798 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:07:54.380868 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 10:07:54.380920 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":101,"threads_started":1}
I20260808 10:07:54.381795 7419 heartbeater.cc:347] Connected to a master server at 127.2.30.190:41797
I20260808 10:07:54.381912 7419 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:54.382073 7419 heartbeater.cc:511] Master 127.2.30.190:41797 requested a full tablet report, sending...
I20260808 10:07:54.382544 6956 ts_manager.cc:194] Registered new tserver with Master: 38a14dcd7cd84b6eba75c1dd9a4546b7 (127.2.30.131:40515)
I20260808 10:07:54.386746 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 7288
I20260808 10:07:54.386831 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:07:54.387036 2170 external_mini_cluster.cc:1428] Setting key 965e7590d0c5d27d7d9f91cb74b4acd8
I20260808 10:07:54.388043 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:07:54.391021 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.389628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37950 (local address 127.2.30.190:41797)
0808 10:07:54.389722 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:54.389724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:54.389945 (+ 221us) server_negotiation.cc:407] Connection header received
0808 10:07:54.389990 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:54.389991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:54.389996 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:54.390041 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:54.390747 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:54.390749 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:54.390749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:54.390752 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:54.390764 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:54.390768 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:54.390815 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:54.390874 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 10:07:54.390886 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:07:54.392803 6956 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:07:54.394464 6956 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:07:54.398464 7429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.396762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:43711 (local address 127.0.0.1:56818)
0808 10:07:54.397093 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:54.397138 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:54.397213 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:54.397378 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:54.397385 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:54.397392 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:54.397983 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:07:54.397984 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:54.398001 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:07:54.398025 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:54.398294 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:54.398302 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:54.398304 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:07:54.398375 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 10:07:54.398390 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":60,"threads_started":1}
I20260808 10:07:54.398694 7428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.396770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:38955 (local address 127.0.0.1:57538)
0808 10:07:54.397291 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:54.397302 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:54.397341 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:54.397930 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:54.397931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:54.397933 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:54.398313 (+ 380us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:07:54.398313 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:54.398320 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:07:54.398331 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:54.398534 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:54.398535 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:54.398536 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 10:07:54.398593 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 10:07:54.398606 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":93,"threads_started":1}
I20260808 10:07:54.398708 7430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.396880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57538 (local address 127.2.30.130:38955)
0808 10:07:54.397356 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:54.397360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:54.397382 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:07:54.397409 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:54.397411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:54.397420 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:54.397483 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:54.398413 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:54.398419 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:54.398420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:54.398424 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:54.398456 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:54.398479 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:54.398540 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:54.398596 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 10:07:54.398613 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":97,"threads_started":1}
I20260808 10:07:54.399197 7431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.396876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56818 (local address 127.2.30.129:43711)
0808 10:07:54.397087 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:54.397092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:54.397216 (+ 124us) server_negotiation.cc:407] Connection header received
0808 10:07:54.397275 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:54.397278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:54.397289 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:54.397374 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:54.398114 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:54.398124 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:54.398124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:54.398129 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:54.398164 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:54.398183 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:54.398262 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:54.398394 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 10:07:54.398413 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":102,"threads_started":1}
I20260808 10:07:54.399209 7432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.396975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:40515 (local address 127.0.0.1:32932)
0808 10:07:54.397811 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:54.397819 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:54.397863 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:54.398308 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:54.398309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:54.398311 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:54.398813 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:07:54.398814 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:54.398819 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:07:54.398830 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:54.399097 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:54.399098 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:54.399099 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 10:07:54.399142 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 10:07:54.399154 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":51,"threads_started":1}
I20260808 10:07:54.399479 7433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.397097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32932 (local address 127.2.30.131:40515)
0808 10:07:54.397284 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:54.397287 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:54.397904 (+ 617us) server_negotiation.cc:407] Connection header received
0808 10:07:54.397941 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:54.397944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:54.397954 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:54.398029 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:54.398916 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:54.398925 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:54.398925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:54.398931 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:54.398996 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:54.399023 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:54.399074 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:54.399317 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 10:07:54.399336 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260808 10:07:54.399484 7218 tablet_service.cc:1467] Processing CreateTablet for tablet 905508ea61af47b7b6b64bdbc7c9773f (DEFAULT_TABLE table=test-table [id=c85ca59a39bf410f910c5ed2673f9bc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:54.399847 7218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 905508ea61af47b7b6b64bdbc7c9773f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:54.399823 7081 tablet_service.cc:1467] Processing CreateTablet for tablet 905508ea61af47b7b6b64bdbc7c9773f (DEFAULT_TABLE table=test-table [id=c85ca59a39bf410f910c5ed2673f9bc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:54.400167 7081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 905508ea61af47b7b6b64bdbc7c9773f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:54.402530 7435 tablet_bootstrap.cc:492] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8: Bootstrap starting.
I20260808 10:07:54.402681 7434 tablet_bootstrap.cc:492] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7: Bootstrap starting.
I20260808 10:07:54.403285 7435 tablet_bootstrap.cc:654] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:54.403431 7434 tablet_bootstrap.cc:654] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:54.403497 7352 tablet_service.cc:1467] Processing CreateTablet for tablet 905508ea61af47b7b6b64bdbc7c9773f (DEFAULT_TABLE table=test-table [id=c85ca59a39bf410f910c5ed2673f9bc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:07:54.403630 7435 log.cc:824] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:54.403738 7434 log.cc:824] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:54.403869 7352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 905508ea61af47b7b6b64bdbc7c9773f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:54.404565 7434 tablet_bootstrap.cc:492] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7: No bootstrap required, opened a new log
I20260808 10:07:54.404565 7435 tablet_bootstrap.cc:492] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8: No bootstrap required, opened a new log
I20260808 10:07:54.404639 7435 ts_tablet_manager.cc:1397] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:07:54.404639 7434 ts_tablet_manager.cc:1397] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:07:54.406442 7438 tablet_bootstrap.cc:492] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7: Bootstrap starting.
I20260808 10:07:54.407179 7438 tablet_bootstrap.cc:654] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:54.407486 7438 log.cc:824] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:54.407385 7435 raft_consensus.cc:348] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4204a91d17d54c6b8060f7da6c740da8" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43711 } } peers { permanent_uuid: "36f2d19ec8e44ea8b5d2305809b0afb7" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38955 } } peers { permanent_uuid: "38a14dcd7cd84b6eba75c1dd9a4546b7" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40515 } }
I20260808 10:07:54.407385 7434 raft_consensus.cc:348] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4204a91d17d54c6b8060f7da6c740da8" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43711 } } peers { permanent_uuid: "36f2d19ec8e44ea8b5d2305809b0afb7" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38955 } } peers { permanent_uuid: "38a14dcd7cd84b6eba75c1dd9a4546b7" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40515 } }
I20260808 10:07:54.407543 7435 raft_consensus.cc:374] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:54.407543 7434 raft_consensus.cc:374] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:54.407580 7435 raft_consensus.cc:729] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4204a91d17d54c6b8060f7da6c740da8, State: Initialized, Role: FOLLOWER
I20260808 10:07:54.407580 7434 raft_consensus.cc:729] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f2d19ec8e44ea8b5d2305809b0afb7, State: Initialized, Role: FOLLOWER
I20260808 10:07:54.407725 7435 consensus_queue.cc:260] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4204a91d17d54c6b8060f7da6c740da8" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43711 } } peers { permanent_uuid: "36f2d19ec8e44ea8b5d2305809b0afb7" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38955 } } peers { permanent_uuid: "38a14dcd7cd84b6eba75c1dd9a4546b7" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40515 } }
I20260808 10:07:54.407723 7434 consensus_queue.cc:260] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4204a91d17d54c6b8060f7da6c740da8" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43711 } } peers { permanent_uuid: "36f2d19ec8e44ea8b5d2305809b0afb7" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38955 } } peers { permanent_uuid: "38a14dcd7cd84b6eba75c1dd9a4546b7" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40515 } }
I20260808 10:07:54.408183 7435 ts_tablet_manager.cc:1428] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260808 10:07:54.408183 7434 ts_tablet_manager.cc:1428] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260808 10:07:54.408211 7285 heartbeater.cc:503] Master 127.2.30.190:41797 was elected leader, sending a full tablet report...
I20260808 10:07:54.408303 7151 heartbeater.cc:503] Master 127.2.30.190:41797 was elected leader, sending a full tablet report...
I20260808 10:07:54.409142 7438 tablet_bootstrap.cc:492] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7: No bootstrap required, opened a new log
I20260808 10:07:54.409284 7438 ts_tablet_manager.cc:1397] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 10:07:54.411541 7438 raft_consensus.cc:348] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4204a91d17d54c6b8060f7da6c740da8" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43711 } } peers { permanent_uuid: "36f2d19ec8e44ea8b5d2305809b0afb7" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38955 } } peers { permanent_uuid: "38a14dcd7cd84b6eba75c1dd9a4546b7" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40515 } }
I20260808 10:07:54.411732 7438 raft_consensus.cc:374] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:54.411782 7438 raft_consensus.cc:729] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a14dcd7cd84b6eba75c1dd9a4546b7, State: Initialized, Role: FOLLOWER
I20260808 10:07:54.411922 7438 consensus_queue.cc:260] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4204a91d17d54c6b8060f7da6c740da8" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43711 } } peers { permanent_uuid: "36f2d19ec8e44ea8b5d2305809b0afb7" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38955 } } peers { permanent_uuid: "38a14dcd7cd84b6eba75c1dd9a4546b7" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40515 } }
I20260808 10:07:54.412235 7438 ts_tablet_manager.cc:1428] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:07:54.412276 7419 heartbeater.cc:503] Master 127.2.30.190:41797 was elected leader, sending a full tablet report...
W20260808 10:07:54.503244 7286 tablet.cc:2369] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:07:54.590843 7440 raft_consensus.cc:482] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:07:54.591007 7440 raft_consensus.cc:504] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4204a91d17d54c6b8060f7da6c740da8" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43711 } } peers { permanent_uuid: "36f2d19ec8e44ea8b5d2305809b0afb7" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38955 } } peers { permanent_uuid: "38a14dcd7cd84b6eba75c1dd9a4546b7" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40515 } }
I20260808 10:07:54.591483 7440 leader_election.cc:288] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36f2d19ec8e44ea8b5d2305809b0afb7 (127.2.30.130:38955), 38a14dcd7cd84b6eba75c1dd9a4546b7 (127.2.30.131:40515)
I20260808 10:07:54.593305 7430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.591967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:38823 (local address 127.2.30.130:38955)
0808 10:07:54.592027 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:54.592029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:54.592040 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:07:54.592078 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:54.592079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:54.592085 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:54.592160 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:54.592919 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:54.592921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:54.592921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:54.592925 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:54.592956 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:54.592962 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:54.593027 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:54.593173 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 10:07:54.593203 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:07:54.593309 7153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.591708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:38955 (local address 127.2.30.129:38823)
0808 10:07:54.591803 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:54.591818 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:54.591873 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:54.592224 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:54.592225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:54.592229 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:54.592786 (+ 557us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:07:54.592787 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:54.592796 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:07:54.592813 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:54.593078 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:54.593079 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:54.593080 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 10:07:54.593157 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 10:07:54.593235 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 10:07:54.593371 7433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.591985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:45089 (local address 127.2.30.131:40515)
0808 10:07:54.592050 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:54.592053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:54.592137 (+ 84us) server_negotiation.cc:407] Connection header received
0808 10:07:54.592186 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:54.592187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:54.592192 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:54.592267 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:54.593118 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:54.593120 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:54.593120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:54.593126 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:54.593147 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:54.593152 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:54.593210 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:54.593278 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 10:07:54.593291 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:07:54.593592 7443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.591845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:40515 (local address 127.2.30.129:45089)
0808 10:07:54.592051 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:54.592062 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:54.592125 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:54.592272 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:54.592273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:54.592276 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:54.592986 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:07:54.592987 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:54.592994 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:07:54.593007 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:54.593222 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:54.593223 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:54.593224 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 10:07:54.593277 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 10:07:54.593290 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":103,"threads_started":1}
I20260808 10:07:54.593706 7238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "905508ea61af47b7b6b64bdbc7c9773f" candidate_uuid: "4204a91d17d54c6b8060f7da6c740da8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36f2d19ec8e44ea8b5d2305809b0afb7" is_pre_election: true
I20260808 10:07:54.593911 7238 raft_consensus.cc:2454] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4204a91d17d54c6b8060f7da6c740da8 in term 0.
I20260808 10:07:54.594262 7036 leader_election.cc:302] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36f2d19ec8e44ea8b5d2305809b0afb7, 4204a91d17d54c6b8060f7da6c740da8; no voters:
I20260808 10:07:54.594409 7372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "905508ea61af47b7b6b64bdbc7c9773f" candidate_uuid: "4204a91d17d54c6b8060f7da6c740da8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a14dcd7cd84b6eba75c1dd9a4546b7" is_pre_election: true
I20260808 10:07:54.594437 7440 raft_consensus.cc:2781] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:07:54.594511 7440 raft_consensus.cc:482] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:07:54.594552 7440 raft_consensus.cc:3037] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:54.594565 7372 raft_consensus.cc:2454] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4204a91d17d54c6b8060f7da6c740da8 in term 0.
I20260808 10:07:54.595461 7440 raft_consensus.cc:504] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4204a91d17d54c6b8060f7da6c740da8" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43711 } } peers { permanent_uuid: "36f2d19ec8e44ea8b5d2305809b0afb7" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38955 } } peers { permanent_uuid: "38a14dcd7cd84b6eba75c1dd9a4546b7" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40515 } }
I20260808 10:07:54.595597 7440 leader_election.cc:288] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [CANDIDATE]: Term 1 election: Requested vote from peers 36f2d19ec8e44ea8b5d2305809b0afb7 (127.2.30.130:38955), 38a14dcd7cd84b6eba75c1dd9a4546b7 (127.2.30.131:40515)
I20260808 10:07:54.595780 7238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "905508ea61af47b7b6b64bdbc7c9773f" candidate_uuid: "4204a91d17d54c6b8060f7da6c740da8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36f2d19ec8e44ea8b5d2305809b0afb7"
I20260808 10:07:54.595829 7372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "905508ea61af47b7b6b64bdbc7c9773f" candidate_uuid: "4204a91d17d54c6b8060f7da6c740da8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38a14dcd7cd84b6eba75c1dd9a4546b7"
I20260808 10:07:54.595887 7238 raft_consensus.cc:3037] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:54.595888 7372 raft_consensus.cc:3037] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:54.596654 7372 raft_consensus.cc:2454] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4204a91d17d54c6b8060f7da6c740da8 in term 1.
I20260808 10:07:54.596765 7238 raft_consensus.cc:2454] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4204a91d17d54c6b8060f7da6c740da8 in term 1.
I20260808 10:07:54.596827 7036 leader_election.cc:302] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38a14dcd7cd84b6eba75c1dd9a4546b7, 4204a91d17d54c6b8060f7da6c740da8; no voters:
I20260808 10:07:54.596915 7440 raft_consensus.cc:2781] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:54.597098 7440 raft_consensus.cc:686] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [term 1 LEADER]: Becoming Leader. State: Replica: 4204a91d17d54c6b8060f7da6c740da8, State: Running, Role: LEADER
I20260808 10:07:54.597208 7440 consensus_queue.cc:237] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4204a91d17d54c6b8060f7da6c740da8" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43711 } } peers { permanent_uuid: "36f2d19ec8e44ea8b5d2305809b0afb7" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38955 } } peers { permanent_uuid: "38a14dcd7cd84b6eba75c1dd9a4546b7" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40515 } }
I20260808 10:07:54.598124 6956 catalog_manager.cc:5495] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4204a91d17d54c6b8060f7da6c740da8 (127.2.30.129). New cstate: current_term: 1 leader_uuid: "4204a91d17d54c6b8060f7da6c740da8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4204a91d17d54c6b8060f7da6c740da8" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 43711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36f2d19ec8e44ea8b5d2305809b0afb7" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38a14dcd7cd84b6eba75c1dd9a4546b7" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40515 } health_report { overall_health: UNKNOWN } } }
W20260808 10:07:54.626482 7152 tablet.cc:2369] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:07:54.627794 7420 tablet.cc:2369] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:07:54.634243 7431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.633266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56820 (local address 127.2.30.129:43711)
0808 10:07:54.633348 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:54.633350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:54.633359 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:07:54.633418 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:54.633418 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:54.633433 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:54.633485 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:54.634083 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:54.634084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:54.634085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:54.634087 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:54.634098 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:54.634101 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:54.634142 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:54.634182 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 10:07:54.634192 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:07:54.636240 7372 raft_consensus.cc:1261] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7 [term 1 FOLLOWER]: Refusing update from remote peer 4204a91d17d54c6b8060f7da6c740da8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:54.636260 7238 raft_consensus.cc:1261] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7 [term 1 FOLLOWER]: Refusing update from remote peer 4204a91d17d54c6b8060f7da6c740da8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:07:54.636612 7440 consensus_queue.cc:1035] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38a14dcd7cd84b6eba75c1dd9a4546b7" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:54.636754 7444 consensus_queue.cc:1035] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36f2d19ec8e44ea8b5d2305809b0afb7" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 38955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:07:54.639038 7447 mvcc.cc:205] Tried to move back new op lower bound from 7316208331306840064 to 7316208331150774272. Current Snapshot: MvccSnapshot[applied={T|T < 7316208331306840064}]
I20260808 10:07:54.639143 7449 mvcc.cc:205] Tried to move back new op lower bound from 7316208331306840064 to 7316208331150774272. Current Snapshot: MvccSnapshot[applied={T|T < 7316208331306840064}]
I20260808 10:07:54.640196 7448 mvcc.cc:205] Tried to move back new op lower bound from 7316208331306840064 to 7316208331150774272. Current Snapshot: MvccSnapshot[applied={T|T < 7316208331306840064}]
I20260808 10:07:54.641594 6956 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:07:54.641683 6956 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:07:54.642462 6956 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad4fdfc15b3a499c86b4fed5d0b9e553: Sending DeleteTablet for 3 replicas of tablet 905508ea61af47b7b6b64bdbc7c9773f
I20260808 10:07:54.642845 7352 tablet_service.cc:1514] Processing DeleteTablet for tablet 905508ea61af47b7b6b64bdbc7c9773f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:54 UTC) from {username='slave'} at 127.0.0.1:32932
I20260808 10:07:54.642827 7218 tablet_service.cc:1514] Processing DeleteTablet for tablet 905508ea61af47b7b6b64bdbc7c9773f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:54 UTC) from {username='slave'} at 127.0.0.1:57538
I20260808 10:07:54.643088 7458 tablet_replica.cc:330] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7: stopping tablet replica
I20260808 10:07:54.643121 7081 tablet_service.cc:1514] Processing DeleteTablet for tablet 905508ea61af47b7b6b64bdbc7c9773f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:07:54 UTC) from {username='slave'} at 127.0.0.1:56818
I20260808 10:07:54.643193 7458 raft_consensus.cc:2229] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:07:54.643288 7458 raft_consensus.cc:2258] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:54.643379 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 7019
I20260808 10:07:54.643414 7459 tablet_replica.cc:330] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8: stopping tablet replica
I20260808 10:07:54.643450 7457 tablet_replica.cc:330] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7: stopping tablet replica
I20260808 10:07:54.643498 7459 raft_consensus.cc:2229] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:07:54.643539 7457 raft_consensus.cc:2229] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:07:54.643581 7459 raft_consensus.cc:2258] T 905508ea61af47b7b6b64bdbc7c9773f P 4204a91d17d54c6b8060f7da6c740da8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:54.643627 7457 raft_consensus.cc:2258] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:07:54.644207 7458 ts_tablet_manager.cc:1905] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:54.645161 7457 ts_tablet_manager.cc:1905] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:07:54.646054 7458 ts_tablet_manager.cc:1918] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:07:54.646107 7458 log.cc:1196] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/905508ea61af47b7b6b64bdbc7c9773f
I20260808 10:07:54.646307 7458 ts_tablet_manager.cc:1939] T 905508ea61af47b7b6b64bdbc7c9773f P 38a14dcd7cd84b6eba75c1dd9a4546b7: Deleting consensus metadata
I20260808 10:07:54.646693 6942 catalog_manager.cc:4841] TS 38a14dcd7cd84b6eba75c1dd9a4546b7 (127.2.30.131:40515): tablet 905508ea61af47b7b6b64bdbc7c9773f (table test-table [id=c85ca59a39bf410f910c5ed2673f9bc9]) successfully deleted
I20260808 10:07:54.647562 7457 ts_tablet_manager.cc:1918] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:07:54.647643 7457 log.cc:1196] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/905508ea61af47b7b6b64bdbc7c9773f
I20260808 10:07:54.647967 7457 ts_tablet_manager.cc:1939] T 905508ea61af47b7b6b64bdbc7c9773f P 36f2d19ec8e44ea8b5d2305809b0afb7: Deleting consensus metadata
I20260808 10:07:54.648417 6942 catalog_manager.cc:4841] TS 36f2d19ec8e44ea8b5d2305809b0afb7 (127.2.30.130:38955): tablet 905508ea61af47b7b6b64bdbc7c9773f (table test-table [id=c85ca59a39bf410f910c5ed2673f9bc9]) successfully deleted
W20260808 10:07:54.654147 6940 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:43711 (error 108)
I20260808 10:07:54.654363 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 7154
I20260808 10:07:54.654798 7432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.654610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:43711 (local address 127.0.0.1:56826)
0808 10:07:54.654686 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:54.654726 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:43711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 10:07:54.655604 6940 catalog_manager.cc:4584] TS 4204a91d17d54c6b8060f7da6c740da8 (127.2.30.129:43711): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 905508ea61af47b7b6b64bdbc7c9773f: Network error: Client connection negotiation failed: client connection to 127.2.30.129:43711: connect: Connection refused (error 111)
I20260808 10:07:54.660112 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 7288
I20260808 10:07:54.666189 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 6925
2026-08-08T10:07:54Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (840 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 10:07:54.687922 2170 mini_oidc.cc:152] Starting JWKS server
I20260808 10:07:54.689131 2170 webserver.cc:460] Webserver started at https://127.0.0.1:44823/ using document root <none> and password file <none>
I20260808 10:07:54.689265 2170 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 10:07:54.689463 2170 webserver.cc:460] Webserver started at http://127.0.0.1:46415/ using document root <none> and password file <none>
2026-08-08T10:07:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:54Z Disabled control of system clock
I20260808 10:07:54.698799 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:45319
--webserver_interface=127.2.30.190
--webserver_port=0
--builtin_ntp_servers=127.2.30.148:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:45319
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44823/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/testchainca.cert with env {}
W20260808 10:07:54.784236 7466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:54.784473 7466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:54.784498 7466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:54.785674 7466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:54.785713 7466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:54.785727 7466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:54.785739 7466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:54.785758 7466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:07:54.785770 7466 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:07:54.785782 7466 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44823/jwks.json
I20260808 10:07:54.787402 7466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:45319
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:45319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44823/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:54.787631 7466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:54.787809 7466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:54.793102 7471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:54.793102 7474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:54.793123 7472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:54.793596 7466 server_base.cc:1034] running on GCE node
I20260808 10:07:54.793936 7466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:54.794265 7466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:54.795425 7466 hybrid_clock.cc:648] HybridClock initialized: now 1786183674795400 us; error 37 us; skew 500 ppm
I20260808 10:07:54.796826 7466 webserver.cc:460] Webserver started at http://127.2.30.190:37191/ using document root <none> and password file <none>
I20260808 10:07:54.797096 7466 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:54.797148 7466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:54.797272 7466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:54.798491 7466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "04fd54f72d694d8db35ccf89aec8ecb8"
format_stamp: "Formatted at 2026-08-08 10:07:54 on dist-test-slave-w206"
server_key: "30e877fdc0a755b66cd4edffcb01d348"
server_key_iv: "4bfd8e83027281aa63fbdc9ad2e0f27c"
server_key_version: "encryptionkey@0"
I20260808 10:07:54.798913 7466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "04fd54f72d694d8db35ccf89aec8ecb8"
format_stamp: "Formatted at 2026-08-08 10:07:54 on dist-test-slave-w206"
server_key: "30e877fdc0a755b66cd4edffcb01d348"
server_key_iv: "4bfd8e83027281aa63fbdc9ad2e0f27c"
server_key_version: "encryptionkey@0"
I20260808 10:07:54.800747 7466 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:07:54.801759 7480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:54.802127 7466 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:54.802220 7466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "04fd54f72d694d8db35ccf89aec8ecb8"
format_stamp: "Formatted at 2026-08-08 10:07:54 on dist-test-slave-w206"
server_key: "30e877fdc0a755b66cd4edffcb01d348"
server_key_iv: "4bfd8e83027281aa63fbdc9ad2e0f27c"
server_key_version: "encryptionkey@0"
I20260808 10:07:54.802349 7466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:54.814028 7466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:54.814419 7466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:54.814567 7466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:54.818899 7466 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:45319
I20260808 10:07:54.819046 7534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:45319 every 8 connection(s)
I20260808 10:07:54.819402 7466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:54.820490 7535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:54.822947 7535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8: Bootstrap starting.
I20260808 10:07:54.823679 7535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:54.823843 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 7466
I20260808 10:07:54.823976 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:07:54.824091 7535 log.cc:824] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:54.824199 2170 external_mini_cluster.cc:1428] Setting key 1ac25dd7ea8d7f9c46fec7d5e12bf962
I20260808 10:07:54.825052 7535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8: No bootstrap required, opened a new log
I20260808 10:07:54.827826 7535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fd54f72d694d8db35ccf89aec8ecb8" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45319 } }
I20260808 10:07:54.827970 7535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:54.827993 7535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04fd54f72d694d8db35ccf89aec8ecb8, State: Initialized, Role: FOLLOWER
I20260808 10:07:54.828152 7535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fd54f72d694d8db35ccf89aec8ecb8" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45319 } }
I20260808 10:07:54.828227 7535 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:54.828251 7535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:54.828303 7535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:54.829046 7535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fd54f72d694d8db35ccf89aec8ecb8" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45319 } }
I20260808 10:07:54.829162 7535 leader_election.cc:302] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04fd54f72d694d8db35ccf89aec8ecb8; no voters:
I20260808 10:07:54.829361 7535 leader_election.cc:288] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:54.829432 7541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:54.829584 7541 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [term 1 LEADER]: Becoming Leader. State: Replica: 04fd54f72d694d8db35ccf89aec8ecb8, State: Running, Role: LEADER
I20260808 10:07:54.829649 7535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:54.829684 7541 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fd54f72d694d8db35ccf89aec8ecb8" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45319 } }
I20260808 10:07:54.830046 7541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04fd54f72d694d8db35ccf89aec8ecb8. Latest consensus state: current_term: 1 leader_uuid: "04fd54f72d694d8db35ccf89aec8ecb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fd54f72d694d8db35ccf89aec8ecb8" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45319 } } }
I20260808 10:07:54.830173 7541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:54.830058 7542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04fd54f72d694d8db35ccf89aec8ecb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fd54f72d694d8db35ccf89aec8ecb8" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45319 } } }
I20260808 10:07:54.830555 7542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:54.830950 7548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:54.831462 7540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.824721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.2.30.190:45319)
0808 10:07:54.824946 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:54.824955 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:54.824989 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:07:54.825558 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:54.825568 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:54.825649 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:54.825811 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:54.826044 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:54.827544 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:54.828684 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:54.829355 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:54.829492 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:54.830425 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:54.830429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:54.830430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:54.830666 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:54.830715 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:54.830733 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:54.830800 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:54.831158 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 10:07:54.831248 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260808 10:07:54.831519 7548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:54.832546 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:54.833426 7548 catalog_manager.cc:1347] Generated new cluster ID: 03ca9e9c1cd348adbe7c2833929e1ea8
I20260808 10:07:54.833483 7548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:54.842721 7548 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:54.843405 7548 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:54.861220 7548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04fd54f72d694d8db35ccf89aec8ecb8: Generated new TSK 0
I20260808 10:07:54.861542 7548 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 10:07:54.898904 7566 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
I20260808 10:07:54.899181 7540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:54.875619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51348 (local address 127.2.30.190:45319)
0808 10:07:54.875692 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:54.875695 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:54.875864 (+ 169us) server_negotiation.cc:407] Connection header received
0808 10:07:54.875986 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:54.875987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:54.875997 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:54.876094 (+ 97us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:07:54.876295 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:54.876792 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:54.877995 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:54.878294 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:54.878372 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:54.878609 (+ 237us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:07:54.898830 (+ 20221us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 10:07:54.899071 (+ 241us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51348: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 10:07:54.899283 7540 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51348: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
E20260808 10:07:54.899219 7566 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260808 10:07:54.899816 7565 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:54.875533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:45319 (local address 127.0.0.1:51348)
0808 10:07:54.875779 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:54.875792 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:54.875880 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:54.876110 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:54.876111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:54.876113 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:07:54.876180 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:54.876182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:54.876893 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:54.876894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:54.877875 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:54.877878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:54.878391 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:54.878392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:54.878463 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:54.878489 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:07:54.899024 (+ 20535us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 10:07:54.899158 (+ 134us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.30.190:45319: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":115,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.2.30.190:45319: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44823/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 10:07:54.900333 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 7466
2026-08-08T10:07:54Z chronyd exiting
I20260808 10:07:54.939949 2170 test_util.cc:175] -----------------------------------------------
I20260808 10:07:54.940059 2170 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786183651681294-2170-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (253 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 10:07:54.940902 2170 mini_oidc.cc:152] Starting JWKS server
I20260808 10:07:54.941625 2170 webserver.cc:460] Webserver started at https://127.0.0.1:45149/ using document root <none> and password file <none>
I20260808 10:07:54.941694 2170 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 10:07:54.941879 2170 webserver.cc:460] Webserver started at http://127.0.0.1:45029/ using document root <none> and password file <none>
2026-08-08T10:07:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:54Z Disabled control of system clock
I20260808 10:07:54.950966 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:45751
--webserver_interface=127.2.30.190
--webserver_port=0
--builtin_ntp_servers=127.2.30.148:41205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:45751
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45149/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/testchainca.cert with env {}
W20260808 10:07:55.037983 7572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:55.038200 7572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:55.038226 7572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:55.039477 7572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:55.039523 7572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:55.039536 7572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:55.039549 7572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:55.039563 7572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:07:55.039577 7572 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:07:55.039587 7572 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45149/jwks.json
I20260808 10:07:55.041209 7572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:41205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:45751
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:45751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45149/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:55.041428 7572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:55.041667 7572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:55.046655 7580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:55.046728 7578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:55.046655 7577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:55.046859 7572 server_base.cc:1034] running on GCE node
I20260808 10:07:55.047179 7572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:55.047484 7572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:55.048656 7572 hybrid_clock.cc:648] HybridClock initialized: now 1786183675048661 us; error 48 us; skew 500 ppm
I20260808 10:07:55.050100 7572 webserver.cc:460] Webserver started at http://127.2.30.190:32845/ using document root <none> and password file <none>
I20260808 10:07:55.050379 7572 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:55.050460 7572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:55.050613 7572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:55.051786 7572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "03d1a7645a4d488492837d8dc7bad5b9"
format_stamp: "Formatted at 2026-08-08 10:07:55 on dist-test-slave-w206"
server_key: "7fd5697a37ece9a7ee51225d6c89209f"
server_key_iv: "ee8b905a568ff36ec35571a1ccbcd11a"
server_key_version: "encryptionkey@0"
I20260808 10:07:55.052145 7572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "03d1a7645a4d488492837d8dc7bad5b9"
format_stamp: "Formatted at 2026-08-08 10:07:55 on dist-test-slave-w206"
server_key: "7fd5697a37ece9a7ee51225d6c89209f"
server_key_iv: "ee8b905a568ff36ec35571a1ccbcd11a"
server_key_version: "encryptionkey@0"
I20260808 10:07:55.053550 7572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 10:07:55.054462 7586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:55.054678 7572 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:07:55.054812 7572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "03d1a7645a4d488492837d8dc7bad5b9"
format_stamp: "Formatted at 2026-08-08 10:07:55 on dist-test-slave-w206"
server_key: "7fd5697a37ece9a7ee51225d6c89209f"
server_key_iv: "ee8b905a568ff36ec35571a1ccbcd11a"
server_key_version: "encryptionkey@0"
I20260808 10:07:55.054893 7572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:55.064618 7572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:55.064965 7572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:55.065141 7572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:55.069434 7572 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:45751
I20260808 10:07:55.069481 7640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:45751 every 8 connection(s)
I20260808 10:07:55.069862 7572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:55.071070 7641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:55.073433 7641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9: Bootstrap starting.
I20260808 10:07:55.074038 7641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:55.074349 7641 log.cc:824] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:55.075132 7641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9: No bootstrap required, opened a new log
I20260808 10:07:55.076287 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 7572
I20260808 10:07:55.076406 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:07:55.076607 2170 external_mini_cluster.cc:1428] Setting key 55ff43501dc6c38dc47b087746a30ab5
I20260808 10:07:55.077214 7641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d1a7645a4d488492837d8dc7bad5b9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45751 } }
I20260808 10:07:55.077412 7641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:55.077441 7641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03d1a7645a4d488492837d8dc7bad5b9, State: Initialized, Role: FOLLOWER
I20260808 10:07:55.077598 7641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d1a7645a4d488492837d8dc7bad5b9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45751 } }
I20260808 10:07:55.077678 7641 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:55.077701 7641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:55.077768 7641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:55.078441 7641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d1a7645a4d488492837d8dc7bad5b9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45751 } }
I20260808 10:07:55.078531 7641 leader_election.cc:302] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03d1a7645a4d488492837d8dc7bad5b9; no voters:
I20260808 10:07:55.078707 7641 leader_election.cc:288] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:55.078755 7646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:55.078923 7646 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [term 1 LEADER]: Becoming Leader. State: Replica: 03d1a7645a4d488492837d8dc7bad5b9, State: Running, Role: LEADER
I20260808 10:07:55.079046 7641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:55.079073 7646 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d1a7645a4d488492837d8dc7bad5b9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45751 } }
I20260808 10:07:55.079423 7646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03d1a7645a4d488492837d8dc7bad5b9. Latest consensus state: current_term: 1 leader_uuid: "03d1a7645a4d488492837d8dc7bad5b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d1a7645a4d488492837d8dc7bad5b9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45751 } } }
I20260808 10:07:55.079499 7646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:55.079437 7648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03d1a7645a4d488492837d8dc7bad5b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d1a7645a4d488492837d8dc7bad5b9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45751 } } }
I20260808 10:07:55.079672 7652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:55.079681 7648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:55.080206 7652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:55.081856 7652 catalog_manager.cc:1347] Generated new cluster ID: 96754871de6a46fea605dee9da04c1fe
I20260808 10:07:55.081909 7652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:55.083822 7645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:55.077193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.2.30.190:45751)
0808 10:07:55.077471 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:55.077480 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:55.077522 (+ 42us) server_negotiation.cc:407] Connection header received
0808 10:07:55.077632 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:55.077637 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:55.077718 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:55.077864 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:55.078113 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.079909 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.081168 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.081769 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.081887 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:55.082839 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:55.082843 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:55.082843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:55.083098 (+ 255us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:55.083143 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:55.083159 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:55.083221 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:55.083505 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 10:07:55.083585 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":97,"threads_started":1}
I20260808 10:07:55.084798 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:55.106891 7652 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:55.107561 7652 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:55.122058 7652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03d1a7645a4d488492837d8dc7bad5b9: Generated new TSK 0
I20260808 10:07:55.122326 7652 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 10:07:55.145678 7672 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 10:07:55.145824 7672 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 10:07:55.146130 7645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:55.131225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43098 (local address 127.2.30.190:45751)
0808 10:07:55.131290 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:55.131292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:55.131404 (+ 112us) server_negotiation.cc:407] Connection header received
0808 10:07:55.131485 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:55.131486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:55.131501 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:55.131598 (+ 97us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:07:55.131792 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.132544 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.133380 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.133687 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.133771 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:55.133933 (+ 162us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:07:55.145873 (+ 11940us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 10:07:55.146049 (+ 176us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43098: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 10:07:55.146198 7645 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43098: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 10:07:55.146263 7671 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:55.131140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:45751 (local address 127.0.0.1:43098)
0808 10:07:55.131322 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:55.131334 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:55.131403 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:55.131617 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:55.131618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:55.131621 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:07:55.131699 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:55.131702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.132654 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.132656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:55.133284 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:55.133286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.133771 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.133772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:55.133822 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:55.133827 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:07:55.145993 (+ 12166us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 10:07:55.146111 (+ 118us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.30.190:45751: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45149/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":126,"threads_started":1}
I20260808 10:07:55.146556 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 7572
2026-08-08T10:07:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (245 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 10:07:55.186510 2170 mini_oidc.cc:152] Starting JWKS server
I20260808 10:07:55.187342 2170 webserver.cc:460] Webserver started at https://127.0.0.1:45231/ using document root <none> and password file <none>
I20260808 10:07:55.187418 2170 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 10:07:55.187592 2170 webserver.cc:460] Webserver started at http://127.0.0.1:41207/ using document root <none> and password file <none>
2026-08-08T10:07:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:55Z Disabled control of system clock
I20260808 10:07:55.196449 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:41011
--webserver_interface=127.2.30.190
--webserver_port=0
--builtin_ntp_servers=127.2.30.148:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:41011
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45231/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 10:07:55.279752 7678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:55.280038 7678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:55.280081 7678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:55.281294 7678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:55.281369 7678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:55.281388 7678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:55.281409 7678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:55.281441 7678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:07:55.281473 7678 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:07:55.281502 7678 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45231/jwks.json
I20260808 10:07:55.282877 7678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:41011
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:41011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45231/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:55.283217 7678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:55.283545 7678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:55.288724 7684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:55.288717 7683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:55.288717 7687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:55.289000 7678 server_base.cc:1034] running on GCE node
I20260808 10:07:55.289265 7678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:55.289565 7678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:55.290776 7678 hybrid_clock.cc:648] HybridClock initialized: now 1786183675290746 us; error 59 us; skew 500 ppm
I20260808 10:07:55.292129 7678 webserver.cc:460] Webserver started at http://127.2.30.190:46507/ using document root <none> and password file <none>
I20260808 10:07:55.292405 7678 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:55.292476 7678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:55.292618 7678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:55.293707 7678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "8a1c5153eeab4acca8e5cd9a3dd5b483"
format_stamp: "Formatted at 2026-08-08 10:07:55 on dist-test-slave-w206"
server_key: "6929caae76f96dec50ec2c450c8588ae"
server_key_iv: "0ec4fe46f83e9f74257fe0ef0c9e3589"
server_key_version: "encryptionkey@0"
I20260808 10:07:55.294096 7678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "8a1c5153eeab4acca8e5cd9a3dd5b483"
format_stamp: "Formatted at 2026-08-08 10:07:55 on dist-test-slave-w206"
server_key: "6929caae76f96dec50ec2c450c8588ae"
server_key_iv: "0ec4fe46f83e9f74257fe0ef0c9e3589"
server_key_version: "encryptionkey@0"
I20260808 10:07:55.295573 7678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 10:07:55.296547 7693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:55.296824 7678 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:55.296952 7678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "8a1c5153eeab4acca8e5cd9a3dd5b483"
format_stamp: "Formatted at 2026-08-08 10:07:55 on dist-test-slave-w206"
server_key: "6929caae76f96dec50ec2c450c8588ae"
server_key_iv: "0ec4fe46f83e9f74257fe0ef0c9e3589"
server_key_version: "encryptionkey@0"
I20260808 10:07:55.297039 7678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:55.307166 7678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:55.307493 7678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:55.307650 7678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:55.311820 7678 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:41011
I20260808 10:07:55.311857 7749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:41011 every 8 connection(s)
I20260808 10:07:55.312218 7678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:55.312898 7750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:55.315819 7750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483: Bootstrap starting.
I20260808 10:07:55.316380 7750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:55.316627 7750 log.cc:824] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:55.317351 7750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483: No bootstrap required, opened a new log
I20260808 10:07:55.318912 7750 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1c5153eeab4acca8e5cd9a3dd5b483" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41011 } }
I20260808 10:07:55.319142 7750 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:55.319171 7750 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a1c5153eeab4acca8e5cd9a3dd5b483, State: Initialized, Role: FOLLOWER
I20260808 10:07:55.319321 7750 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1c5153eeab4acca8e5cd9a3dd5b483" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41011 } }
I20260808 10:07:55.319397 7750 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:55.319422 7750 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:55.319450 7750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:55.319988 7750 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1c5153eeab4acca8e5cd9a3dd5b483" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41011 } }
I20260808 10:07:55.320075 7750 leader_election.cc:302] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a1c5153eeab4acca8e5cd9a3dd5b483; no voters:
I20260808 10:07:55.320210 7750 leader_election.cc:288] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:55.320293 7754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:55.320477 7750 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:55.320480 7754 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [term 1 LEADER]: Becoming Leader. State: Replica: 8a1c5153eeab4acca8e5cd9a3dd5b483, State: Running, Role: LEADER
I20260808 10:07:55.320657 7754 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1c5153eeab4acca8e5cd9a3dd5b483" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41011 } }
I20260808 10:07:55.321280 7755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a1c5153eeab4acca8e5cd9a3dd5b483" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1c5153eeab4acca8e5cd9a3dd5b483" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41011 } } }
I20260808 10:07:55.321346 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 7678
I20260808 10:07:55.321388 7755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:55.321453 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:07:55.321643 2170 external_mini_cluster.cc:1428] Setting key 4303e0845cd347c67ac6066f26afa284
I20260808 10:07:55.321983 7763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:55.322387 7756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a1c5153eeab4acca8e5cd9a3dd5b483. Latest consensus state: current_term: 1 leader_uuid: "8a1c5153eeab4acca8e5cd9a3dd5b483" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a1c5153eeab4acca8e5cd9a3dd5b483" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41011 } } }
I20260808 10:07:55.322474 7756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:55.322548 7763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:55.324308 7763 catalog_manager.cc:1347] Generated new cluster ID: 49ba2ac515774de9a47b22e2343edcb1
I20260808 10:07:55.324371 7763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:55.327711 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:55.322399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57304 (local address 127.2.30.190:41011)
0808 10:07:55.322728 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:55.322739 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:55.322774 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:07:55.322908 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:55.322914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:55.323097 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:55.323632 (+ 535us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:55.323716 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.324869 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.325683 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.326125 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.326267 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:55.327008 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:55.327012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:55.327012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:55.327196 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:55.327229 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:55.327246 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:55.327329 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:55.327412 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 10:07:55.327496 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":89,"threads_started":1}
I20260808 10:07:55.328665 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:55.352182 7763 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:55.352849 7763 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:55.356907 7763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a1c5153eeab4acca8e5cd9a3dd5b483: Generated new TSK 0
I20260808 10:07:55.357120 7763 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:55.374485 7773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:55.370588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57310 (local address 127.2.30.190:41011)
0808 10:07:55.370641 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:55.370643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:55.370768 (+ 125us) server_negotiation.cc:407] Connection header received
0808 10:07:55.370861 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:55.370862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:55.370873 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:55.370939 (+ 66us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:07:55.371175 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.371705 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.372881 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.373070 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.373125 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:55.373335 (+ 210us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:07:55.374175 (+ 840us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45231/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 10:07:55.374411 (+ 236us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57310: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45231/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 10:07:55.374485 7780 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:55.370491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41011 (local address 127.0.0.1:57310)
0808 10:07:55.370689 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:55.370701 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:55.370785 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:55.370968 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:55.370991 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:55.370993 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:07:55.371065 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:55.371068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.371820 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.371821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:55.372793 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:55.372796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.373159 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.373161 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:55.373228 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:55.373234 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:07:55.374288 (+ 1054us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45231/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 10:07:55.374378 (+ 90us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.30.190:41011: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45231/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":106,"threads_started":1}
W20260808 10:07:55.374562 7773 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57310: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45231/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260808 10:07:55.374841 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 7678
E20260808 10:07:55.386229 7781 webserver.cc:533] Webserver: sslize failed:
E20260808 10:07:55.386309 7781 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T10:07:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (227 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T10:07:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:55Z Disabled control of system clock
I20260808 10:07:55.423393 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:34153
--webserver_interface=127.2.30.190
--webserver_port=0
--builtin_ntp_servers=127.2.30.148:33837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:34153
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/testchainca.cert with env {}
W20260808 10:07:55.506712 7785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:55.506997 7785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:55.507081 7785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:55.508301 7785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:55.508383 7785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:55.508404 7785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:55.508436 7785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:55.508486 7785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 10:07:55.508517 7785 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 10:07:55.508543 7785 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 10:07:55.510118 7785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:33837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:34153
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:34153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:55.510418 7785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:55.510659 7785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:07:55.516014 7785 server_base.cc:1034] running on GCE node
W20260808 10:07:55.515952 7790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:55.515960 7793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:55.515973 7791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:55.516426 7785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:55.516746 7785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:55.517948 7785 hybrid_clock.cc:648] HybridClock initialized: now 1786183675517944 us; error 55 us; skew 500 ppm
I20260808 10:07:55.519335 7785 webserver.cc:460] Webserver started at http://127.2.30.190:41793/ using document root <none> and password file <none>
I20260808 10:07:55.519588 7785 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:55.519665 7785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:55.519835 7785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:55.520931 7785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "3c89740ae92a4d058e286329c4996673"
format_stamp: "Formatted at 2026-08-08 10:07:55 on dist-test-slave-w206"
server_key: "ad914c6f9c49414621c90f6989aeee1d"
server_key_iv: "82be685bbf17ba8a7380a1e2b151dd9e"
server_key_version: "encryptionkey@0"
I20260808 10:07:55.521337 7785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "3c89740ae92a4d058e286329c4996673"
format_stamp: "Formatted at 2026-08-08 10:07:55 on dist-test-slave-w206"
server_key: "ad914c6f9c49414621c90f6989aeee1d"
server_key_iv: "82be685bbf17ba8a7380a1e2b151dd9e"
server_key_version: "encryptionkey@0"
I20260808 10:07:55.522665 7785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:07:55.523552 7800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:55.523720 7785 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:07:55.523842 7785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "3c89740ae92a4d058e286329c4996673"
format_stamp: "Formatted at 2026-08-08 10:07:55 on dist-test-slave-w206"
server_key: "ad914c6f9c49414621c90f6989aeee1d"
server_key_iv: "82be685bbf17ba8a7380a1e2b151dd9e"
server_key_version: "encryptionkey@0"
I20260808 10:07:55.523926 7785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:55.543152 7785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:55.543493 7785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:55.543606 7785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:55.547652 7785 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:34153
I20260808 10:07:55.547720 7855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:34153 every 8 connection(s)
I20260808 10:07:55.548084 7785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:55.548308 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 7785
I20260808 10:07:55.548445 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:07:55.548692 2170 external_mini_cluster.cc:1428] Setting key 87bb6645b6636b6c0be32543a384c437
I20260808 10:07:55.549361 7856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:55.552553 7856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673: Bootstrap starting.
I20260808 10:07:55.553335 7856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:55.553627 7856 log.cc:824] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:55.554606 7856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673: No bootstrap required, opened a new log
I20260808 10:07:55.554823 7859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:55.549232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52466 (local address 127.2.30.190:34153)
0808 10:07:55.549489 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:55.549500 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:55.549536 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:07:55.549982 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:55.549989 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:55.550048 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:55.550161 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:55.550482 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.551388 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.552496 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.552995 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.553086 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:55.553955 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:55.553959 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:55.553959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:07:55.554182 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:55.554228 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:07:55.554240 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:55.554295 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:55.554419 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 10:07:55.554518 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":125,"threads_started":1}
I20260808 10:07:55.555939 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:07:55.557155 7856 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c89740ae92a4d058e286329c4996673" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 34153 } }
I20260808 10:07:55.557298 7856 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:55.557322 7856 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c89740ae92a4d058e286329c4996673, State: Initialized, Role: FOLLOWER
I20260808 10:07:55.557461 7856 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c89740ae92a4d058e286329c4996673" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 34153 } }
I20260808 10:07:55.557531 7856 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:55.557576 7856 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:55.557624 7856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:55.558234 7856 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c89740ae92a4d058e286329c4996673" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 34153 } }
I20260808 10:07:55.558322 7856 leader_election.cc:302] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c89740ae92a4d058e286329c4996673; no voters:
I20260808 10:07:55.558523 7856 leader_election.cc:288] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:55.558593 7861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:55.558748 7861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [term 1 LEADER]: Becoming Leader. State: Replica: 3c89740ae92a4d058e286329c4996673, State: Running, Role: LEADER
I20260808 10:07:55.558801 7856 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:55.558843 7861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c89740ae92a4d058e286329c4996673" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 34153 } }
I20260808 10:07:55.559226 7862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c89740ae92a4d058e286329c4996673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c89740ae92a4d058e286329c4996673" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 34153 } } }
I20260808 10:07:55.559230 7863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c89740ae92a4d058e286329c4996673. Latest consensus state: current_term: 1 leader_uuid: "3c89740ae92a4d058e286329c4996673" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c89740ae92a4d058e286329c4996673" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 34153 } } }
I20260808 10:07:55.559386 7862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:55.559396 7863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673 [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:55.559937 7870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:55.560498 7870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:55.562170 7870 catalog_manager.cc:1347] Generated new cluster ID: 9a7e45b6128f442abad3f25657b125e8
I20260808 10:07:55.562232 7870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:55.572152 7870 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:55.572882 7870 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:55.581678 7870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c89740ae92a4d058e286329c4996673: Generated new TSK 0
I20260808 10:07:55.581902 7870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:07:56.031812 7859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:55.588852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52474 (local address 127.2.30.190:34153)
0808 10:07:55.588917 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:55.588920 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:55.589019 (+ 99us) server_negotiation.cc:407] Connection header received
0808 10:07:55.589068 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:55.589069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:55.589084 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:55.589173 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0808 10:07:55.589337 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.590124 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.591010 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.591330 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.591404 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:55.591579 (+ 175us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 10:07:56.031561 (+439982us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 10:07:56.031744 (+ 183us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52474: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 10:07:56.031973 7886 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:55.588740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:34153 (local address 127.0.0.1:52474)
0808 10:07:55.588940 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:55.588951 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:55.589015 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:55.589178 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:55.589179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:55.589181 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 10:07:55.589236 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:55.589237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.590275 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.590277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:55.590896 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:55.590898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:55.591418 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:55.591419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:55.591465 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:55.591471 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 10:07:56.031738 (+440267us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 10:07:56.031879 (+ 141us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.30.190:34153: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":131,"threads_started":1}
W20260808 10:07:56.032011 7859 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52474: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260808 10:07:56.032466 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 7785
2026-08-08T10:07:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (638 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (93 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (95 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:56 dist-test-slave-w206 krb5kdc[7898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:56 dist-test-slave-w206 krb5kdc[7898](info): set up 2 sockets
Aug 08 10:07:56 dist-test-slave-w206 krb5kdc[7898](info): commencing operation
krb5kdc: starting...
W20260808 10:07:57.287846 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:07:57 dist-test-slave-w206 krb5kdc[7898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:07:57.426710 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:46695
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:43955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:46695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:57.511293 7911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:57.511526 7911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:57.511551 7911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:57.512712 7911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:57.512746 7911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:57.512761 7911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:57.512773 7911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:57.512786 7911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:57.514236 7911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:43955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:46695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:46695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:57.514451 7911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:57.514652 7911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:57.520095 7916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:57.520115 7917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:57.520130 7919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:57.520429 7911 server_base.cc:1034] running on GCE node
I20260808 10:07:57.520831 7911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:57.521139 7911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:57.522339 7911 hybrid_clock.cc:648] HybridClock initialized: now 1786183677522322 us; error 32 us; skew 500 ppm
Aug 08 10:07:57 dist-test-slave-w206 krb5kdc[7898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183677, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:57.525408 7911 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:07:57.526028 7911 webserver.cc:460] Webserver started at http://127.2.30.190:45939/ using document root <none> and password file <none>
I20260808 10:07:57.526312 7911 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:57.526393 7911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:57.526566 7911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:57.527915 7911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "22232abe8ad44acc9454a422115e480f"
format_stamp: "Formatted at 2026-08-08 10:07:57 on dist-test-slave-w206"
server_key: "f563062a852e1484040aada7536bff08"
server_key_iv: "ee0b4796961beca47fd3f353f1602d07"
server_key_version: "encryptionkey@0"
I20260808 10:07:57.528324 7911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "22232abe8ad44acc9454a422115e480f"
format_stamp: "Formatted at 2026-08-08 10:07:57 on dist-test-slave-w206"
server_key: "f563062a852e1484040aada7536bff08"
server_key_iv: "ee0b4796961beca47fd3f353f1602d07"
server_key_version: "encryptionkey@0"
I20260808 10:07:57.529857 7911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 10:07:57.530704 7926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:57.530920 7911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:57.531072 7911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "22232abe8ad44acc9454a422115e480f"
format_stamp: "Formatted at 2026-08-08 10:07:57 on dist-test-slave-w206"
server_key: "f563062a852e1484040aada7536bff08"
server_key_iv: "ee0b4796961beca47fd3f353f1602d07"
server_key_version: "encryptionkey@0"
I20260808 10:07:57.531162 7911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:57.547524 7911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:57.549700 7911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:57.549906 7911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:57.554569 7911 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:46695
I20260808 10:07:57.554615 7980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:46695 every 8 connection(s)
I20260808 10:07:57.555088 7911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:57.556046 7981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:57.558357 7981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f: Bootstrap starting.
I20260808 10:07:57.558936 7981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:57.559278 7981 log.cc:824] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:57.560083 7981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f: No bootstrap required, opened a new log
I20260808 10:07:57.561650 7981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22232abe8ad44acc9454a422115e480f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46695 } }
I20260808 10:07:57.561789 7981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:57.561813 7981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22232abe8ad44acc9454a422115e480f, State: Initialized, Role: FOLLOWER
I20260808 10:07:57.561976 7981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22232abe8ad44acc9454a422115e480f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46695 } }
I20260808 10:07:57.562065 7981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:57.562090 7981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:57.562120 7981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:57.562659 7981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22232abe8ad44acc9454a422115e480f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46695 } }
I20260808 10:07:57.562752 7981 leader_election.cc:302] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22232abe8ad44acc9454a422115e480f; no voters:
I20260808 10:07:57.562929 7981 leader_election.cc:288] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:57.562997 7984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:57.563104 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 7911
I20260808 10:07:57.563217 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:07:57.563265 7981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:57.563359 7984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [term 1 LEADER]: Becoming Leader. State: Replica: 22232abe8ad44acc9454a422115e480f, State: Running, Role: LEADER
I20260808 10:07:57.563453 7984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22232abe8ad44acc9454a422115e480f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46695 } }
I20260808 10:07:57.563499 2170 external_mini_cluster.cc:1428] Setting key df492c00af043eae2e20878d7941d522
I20260808 10:07:57.563803 7984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22232abe8ad44acc9454a422115e480f. Latest consensus state: current_term: 1 leader_uuid: "22232abe8ad44acc9454a422115e480f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22232abe8ad44acc9454a422115e480f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46695 } } }
I20260808 10:07:57.563817 7986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22232abe8ad44acc9454a422115e480f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22232abe8ad44acc9454a422115e480f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46695 } } }
I20260808 10:07:57.563913 7984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:57.563934 7986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:57.564628 7994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:57.565289 7994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:07:57.566999 7994 catalog_manager.cc:1347] Generated new cluster ID: b76253f0100d415894ca85410c11efe9
I20260808 10:07:57.567055 7994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:07:57 dist-test-slave-w206 krb5kdc[7898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:57.574030 7995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:57.564209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53338 (local address 127.2.30.190:46695)
0808 10:07:57.564408 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:57.564418 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:57.564462 (+ 44us) server_negotiation.cc:407] Connection header received
0808 10:07:57.564602 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:57.564609 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:57.564677 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:57.565483 (+ 806us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:57.565715 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:57.566931 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:57.567784 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:57.568185 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:57.568315 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:57.570722 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:57.570726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:57.570727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:57.570739 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:57.572447 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:57.572829 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:57.572832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:57.572832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:57.572937 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:57.573150 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:57.573151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:57.573151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:57.573412 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:57.573599 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:57.573669 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 10:07:57.573818 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":81,"threads_started":1}
I20260808 10:07:57.581254 7994 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:57.581811 7994 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:57.584235 7994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22232abe8ad44acc9454a422115e480f: Generated new TSK 0
I20260808 10:07:57.584436 7994 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:07:57.615020 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46695
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:43955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:57.700847 8008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:57.701102 8008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:57.701143 8008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:57.702406 8008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:57.702482 8008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:57.702530 8008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:07:57.704223 8008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:43955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:57.704530 8008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:57.704803 8008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:57.709990 8016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:57.710006 8014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:57.710141 8013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:57.710412 8008 server_base.cc:1034] running on GCE node
I20260808 10:07:57.710631 8008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:57.710915 8008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:57.712126 8008 hybrid_clock.cc:648] HybridClock initialized: now 1786183677712115 us; error 32 us; skew 500 ppm
Aug 08 10:07:57 dist-test-slave-w206 krb5kdc[7898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183677, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:57.714850 8008 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:07:57.715369 8008 webserver.cc:460] Webserver started at http://127.2.30.129:38927/ using document root <none> and password file <none>
I20260808 10:07:57.715566 8008 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:57.715607 8008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:57.715700 8008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:57.716840 8008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "a7d7edf96ce54e7e8439f99d90ba7db2"
format_stamp: "Formatted at 2026-08-08 10:07:57 on dist-test-slave-w206"
server_key: "2f0b0a265a9d029db7fa6efa0b31c26a"
server_key_iv: "5abc21df0ac65d7269526ee392687fde"
server_key_version: "encryptionkey@0"
I20260808 10:07:57.717154 8008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "a7d7edf96ce54e7e8439f99d90ba7db2"
format_stamp: "Formatted at 2026-08-08 10:07:57 on dist-test-slave-w206"
server_key: "2f0b0a265a9d029db7fa6efa0b31c26a"
server_key_iv: "5abc21df0ac65d7269526ee392687fde"
server_key_version: "encryptionkey@0"
I20260808 10:07:57.718536 8008 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:07:57.719383 8023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:57.719581 8008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:07:57.719661 8008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "a7d7edf96ce54e7e8439f99d90ba7db2"
format_stamp: "Formatted at 2026-08-08 10:07:57 on dist-test-slave-w206"
server_key: "2f0b0a265a9d029db7fa6efa0b31c26a"
server_key_iv: "5abc21df0ac65d7269526ee392687fde"
server_key_version: "encryptionkey@0"
I20260808 10:07:57.719761 8008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:57.728647 8008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:57.731328 8008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:57.731504 8008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:57.731778 8008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:57.732121 8008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:57.732156 8008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:57.732179 8008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:57.732191 8008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:57.738498 8008 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:40611
I20260808 10:07:57.738549 8141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:40611 every 8 connection(s)
I20260808 10:07:57.738909 8008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:07:57.741569 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 8008
I20260808 10:07:57.741674 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:07:57.741895 2170 external_mini_cluster.cc:1428] Setting key 0521200c70b728b79dd044d0211be840
Aug 08 10:07:57 dist-test-slave-w206 krb5kdc[7898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183677, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:57.750419 7995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:57.739746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:46601 (local address 127.2.30.190:46695)
0808 10:07:57.739822 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:57.739824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:57.740075 (+ 251us) server_negotiation.cc:407] Connection header received
0808 10:07:57.741035 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:57.741036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:57.741049 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:57.741111 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:57.741791 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:57.742464 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:57.743689 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:57.743916 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:57.743998 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:57.747233 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:57.747235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:57.747235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:57.747243 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:57.748774 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:57.749325 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:57.749326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:57.749326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:57.749397 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:57.749712 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:57.749713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:57.749713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:57.749827 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:57.749880 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:57.750263 (+ 383us) server_negotiation.cc:299] Negotiation successful
0808 10:07:57.750335 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:07:57.750763 8144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:57.739738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46695 (local address 127.2.30.129:46601)
0808 10:07:57.739931 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:57.739977 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:57.740939 (+ 962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:57.741170 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:57.741174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:57.741447 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:57.741686 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:57.741692 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:57.742576 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:57.742577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:57.743598 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:57.743601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:57.744020 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:57.744021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:57.744164 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:57.744867 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:57.744878 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:57.747132 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:57.748869 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:57.748878 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:57.748881 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:57.749244 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:57.749469 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:57.749470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:57.749471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:57.749639 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:57.749895 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:57.749903 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:57.750102 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 10:07:57.750237 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 10:07:57.750484 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260808 10:07:57.751400 8142 heartbeater.cc:347] Connected to a master server at 127.2.30.190:46695
I20260808 10:07:57.751506 8142 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:57.751717 8142 heartbeater.cc:511] Master 127.2.30.190:46695 requested a full tablet report, sending...
I20260808 10:07:57.752280 7943 ts_manager.cc:194] Registered new tserver with Master: a7d7edf96ce54e7e8439f99d90ba7db2 (127.2.30.129:40611)
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
I20260808 10:07:57.752892 7943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:46601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:07:57.774464 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46695
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:43955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:57.859726 8147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:57.860060 8147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:57.860090 8147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:57.861335 8147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:57.861380 8147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:57.861408 8147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:07:57.862874 8147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:43955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:57.863163 8147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:57.863487 8147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:57.868901 8155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:57.868906 8152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:57.869027 8147 server_base.cc:1034] running on GCE node
W20260808 10:07:57.868891 8153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:57.869477 8147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:57.869760 8147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:57.870929 8147 hybrid_clock.cc:648] HybridClock initialized: now 1786183677870924 us; error 37 us; skew 500 ppm
Aug 08 10:07:57 dist-test-slave-w206 krb5kdc[7898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183677, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:57.873634 8147 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:07:57.874166 8147 webserver.cc:460] Webserver started at http://127.2.30.130:40505/ using document root <none> and password file <none>
I20260808 10:07:57.874413 8147 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:57.874483 8147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:57.874640 8147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:57.875828 8147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "e993119400304bd5a9c85e9f8a0a8945"
format_stamp: "Formatted at 2026-08-08 10:07:57 on dist-test-slave-w206"
server_key: "a313475a4a3947ab0bae0c5137c3920e"
server_key_iv: "823dcb58fc61a7314ef16cadfdb30f3c"
server_key_version: "encryptionkey@0"
I20260808 10:07:57.876199 8147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "e993119400304bd5a9c85e9f8a0a8945"
format_stamp: "Formatted at 2026-08-08 10:07:57 on dist-test-slave-w206"
server_key: "a313475a4a3947ab0bae0c5137c3920e"
server_key_iv: "823dcb58fc61a7314ef16cadfdb30f3c"
server_key_version: "encryptionkey@0"
I20260808 10:07:57.877599 8147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:07:57.878351 8162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:57.878537 8147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:07:57.878651 8147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "e993119400304bd5a9c85e9f8a0a8945"
format_stamp: "Formatted at 2026-08-08 10:07:57 on dist-test-slave-w206"
server_key: "a313475a4a3947ab0bae0c5137c3920e"
server_key_iv: "823dcb58fc61a7314ef16cadfdb30f3c"
server_key_version: "encryptionkey@0"
I20260808 10:07:57.878728 8147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:57.898169 8147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:57.900820 8147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:57.901002 8147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:57.901264 8147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:57.901607 8147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:57.901664 8147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:57.901708 8147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:57.901736 8147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:57.908093 8147 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:33085
I20260808 10:07:57.908136 8281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:33085 every 8 connection(s)
I20260808 10:07:57.908486 8147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:07:57.910827 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 8147
I20260808 10:07:57.910956 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:07:57.911192 2170 external_mini_cluster.cc:1428] Setting key 89396d7060136d812184267b1de9b824
Aug 08 10:07:57 dist-test-slave-w206 krb5kdc[7898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183677, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
I20260808 10:07:57.925534 7995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:57.909299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:49269 (local address 127.2.30.190:46695)
0808 10:07:57.909376 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:57.909378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:57.909559 (+ 181us) server_negotiation.cc:407] Connection header received
0808 10:07:57.910518 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:57.910519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:57.910530 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:57.910617 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:57.911587 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:57.912203 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:57.913290 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:57.913620 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:57.913701 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:57.916872 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:57.916873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:57.916873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:57.916882 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:57.923590 (+ 6708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:57.924355 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:57.924357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:57.924357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:57.924445 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:57.924743 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:57.924745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:57.924745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:57.924928 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:57.925003 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:57.925275 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 10:07:57.925383 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:07:57.925652 8284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:57.909244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46695 (local address 127.2.30.130:49269)
0808 10:07:57.909416 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:57.909461 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:57.910416 (+ 955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:57.910717 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:57.910722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:57.911179 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:57.911477 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:57.911483 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:57.912329 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:57.912331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:57.913183 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:57.913185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:57.913785 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:57.913786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:57.913975 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:57.914615 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:57.914643 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:57.916746 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:57.923715 (+ 6969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:57.923732 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:57.923735 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:57.924187 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:57.924526 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:57.924526 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:57.924527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:57.924673 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:57.924996 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:57.925001 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:57.925141 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 10:07:57.925274 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 10:07:57.925429 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260808 10:07:57.926214 8282 heartbeater.cc:347] Connected to a master server at 127.2.30.190:46695
I20260808 10:07:57.926343 8282 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:57.926545 8282 heartbeater.cc:511] Master 127.2.30.190:46695 requested a full tablet report, sending...
I20260808 10:07:57.927001 7943 ts_manager.cc:194] Registered new tserver with Master: e993119400304bd5a9c85e9f8a0a8945 (127.2.30.130:33085)
I20260808 10:07:57.927439 7943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:49269
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:07:57.942121 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46695
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:43955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:58.029909 8287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:58.030133 8287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:58.030157 8287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:58.031286 8287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:58.031322 8287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:58.031347 8287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:07:58.032764 8287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:43955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:58.032961 8287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:58.033149 8287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:58.038668 8292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:58.038689 8293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:58.038932 8287 server_base.cc:1034] running on GCE node
W20260808 10:07:58.038668 8295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:58.039372 8287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:58.039657 8287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:58.040853 8287 hybrid_clock.cc:648] HybridClock initialized: now 1786183678040833 us; error 37 us; skew 500 ppm
Aug 08 10:07:58 dist-test-slave-w206 krb5kdc[7898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183678, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:58.043744 8287 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:07:58.044274 8287 webserver.cc:460] Webserver started at http://127.2.30.131:45869/ using document root <none> and password file <none>
I20260808 10:07:58.044519 8287 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:58.044567 8287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:58.044701 8287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:58.045778 8287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "4cfa1dcbafbc44e8aaa7ac2eedc2f9c3"
format_stamp: "Formatted at 2026-08-08 10:07:58 on dist-test-slave-w206"
server_key: "a8f2299ad2e508773f15012810b45d33"
server_key_iv: "fd46e8cb536c43722a793d0f03ce9482"
server_key_version: "encryptionkey@0"
I20260808 10:07:58.046159 8287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "4cfa1dcbafbc44e8aaa7ac2eedc2f9c3"
format_stamp: "Formatted at 2026-08-08 10:07:58 on dist-test-slave-w206"
server_key: "a8f2299ad2e508773f15012810b45d33"
server_key_iv: "fd46e8cb536c43722a793d0f03ce9482"
server_key_version: "encryptionkey@0"
I20260808 10:07:58.047612 8287 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:07:58.048460 8302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:58.048647 8287 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:07:58.048754 8287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "4cfa1dcbafbc44e8aaa7ac2eedc2f9c3"
format_stamp: "Formatted at 2026-08-08 10:07:58 on dist-test-slave-w206"
server_key: "a8f2299ad2e508773f15012810b45d33"
server_key_iv: "fd46e8cb536c43722a793d0f03ce9482"
server_key_version: "encryptionkey@0"
I20260808 10:07:58.048858 8287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:58.061152 8287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:58.063609 8287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:58.063804 8287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:58.064112 8287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:58.064474 8287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:58.064531 8287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:58.064576 8287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:58.064603 8287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:58.070953 8287 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:46379
I20260808 10:07:58.071024 8421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:46379 every 8 connection(s)
I20260808 10:07:58.071383 8287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
Aug 08 10:07:58 dist-test-slave-w206 krb5kdc[7898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183678, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:07:58.078667 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 8287
I20260808 10:07:58.078809 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:07:58.079061 2170 external_mini_cluster.cc:1428] Setting key 82d803b0f8cf225d153f2b023a9e7719
I20260808 10:07:58.082310 7995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:58.072154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:50781 (local address 127.2.30.190:46695)
0808 10:07:58.072207 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:58.072209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:58.072447 (+ 238us) server_negotiation.cc:407] Connection header received
0808 10:07:58.073416 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:58.073417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:58.073427 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:58.073489 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:58.074145 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:58.074778 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:58.076099 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:58.076281 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:58.076338 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:58.078760 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:58.078762 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:58.078762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:58.078773 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:58.080673 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:58.081210 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:58.081212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:58.081212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:58.081290 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:58.081607 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:58.081608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:58.081608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:58.081740 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:58.081824 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:58.082117 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 10:07:58.082210 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:07:58.082559 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:58.072113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46695 (local address 127.2.30.131:50781)
0808 10:07:58.072301 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:58.072347 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:58.073320 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:58.073547 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:58.073551 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:58.073838 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:58.074056 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:58.074061 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:58.074926 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:58.074927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:58.076005 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:58.076008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:58.076360 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:58.076361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:58.076474 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:58.077032 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:58.077040 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:58.078631 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:58.080764 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:58.080771 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:58.080773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:58.081056 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:58.081366 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:58.081367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:58.081368 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:58.081544 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:58.081816 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:58.081823 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:58.081979 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 10:07:58.082094 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 10:07:58.082280 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":107,"threads_started":1}
I20260808 10:07:58.083076 8422 heartbeater.cc:347] Connected to a master server at 127.2.30.190:46695
I20260808 10:07:58.083253 8422 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:58.083498 8422 heartbeater.cc:511] Master 127.2.30.190:46695 requested a full tablet report, sending...
I20260808 10:07:58.084087 7943 ts_manager.cc:194] Registered new tserver with Master: 4cfa1dcbafbc44e8aaa7ac2eedc2f9c3 (127.2.30.131:46379)
I20260808 10:07:58.084580 7943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:50781
I20260808 10:07:58.091872 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/test-admin.keytab.
Aug 08 10:07:58 dist-test-slave-w206 krb5kdc[7898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:58.108803 2170 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 10:07:58.110956 8429 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:58.109884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51408 (local address 127.2.30.129:40611)
0808 10:07:58.110112 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:58.110117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:58.110129 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:07:58.110157 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:58.110158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:58.110173 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:58.110252 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:58.110862 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51408: BlockingRecv error: recv got EOF from 127.0.0.1:51408 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":120,"threads_started":1}
W20260808 10:07:58.111253 2170 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/krb5cc' of type FILE
Aug 08 10:07:58 dist-test-slave-w206 krb5kdc[7898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:58.112425 2170 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 10:07:58 dist-test-slave-w206 krb5kdc[7898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:58.113548 2170 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 10:07:58 dist-test-slave-w206 krb5kdc[7898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.129@KRBTEST.COM
I20260808 10:07:58.122705 8429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:58.114284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51410 (local address 127.2.30.129:40611)
0808 10:07:58.114361 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:58.114363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:58.114470 (+ 107us) server_negotiation.cc:407] Connection header received
0808 10:07:58.114518 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:58.114518 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:58.114530 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:58.114578 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:58.115071 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:58.115823 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:58.116642 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:58.116982 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:58.117047 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:58.119522 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:58.119534 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:58.119535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:58.119545 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:58.121362 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:58.121753 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:58.121757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:58.121757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:58.121825 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:58.122041 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:58.122042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:58.122042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:58.122231 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:58.122379 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:58.122482 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 10:07:58.122580 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 10:07:58.125119 8429 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:58.124510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51422 (local address 127.2.30.129:40611)
0808 10:07:58.124575 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:58.124577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:58.124637 (+ 60us) server_negotiation.cc:407] Connection header received
0808 10:07:58.124705 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:58.124706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:58.124719 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:58.124768 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:58.125068 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51422: BlockingRecv error: recv got EOF from 127.0.0.1:51422 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 10:07:58.125272 2170 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/krb5cc' of type FILE
Aug 08 10:07:58 dist-test-slave-w206 krb5kdc[7898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:58.126423 2170 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 10:07:58 dist-test-slave-w206 krb5kdc[7898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:58.127599 2170 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 10:07:58 dist-test-slave-w206 krb5kdc[7898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.129@KRBTEST.COM
I20260808 10:07:58.135996 8429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:58.128201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51426 (local address 127.2.30.129:40611)
0808 10:07:58.128261 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:58.128264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:58.128478 (+ 214us) server_negotiation.cc:407] Connection header received
0808 10:07:58.128508 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:58.128509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:58.128521 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:58.128618 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:58.129004 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:58.129520 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:58.130253 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:58.130467 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:58.130552 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:58.132865 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:58.132867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:58.132867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:58.132874 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:58.134462 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:58.134880 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:58.134881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:58.134881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:58.134943 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:58.135359 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:58.135360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:58.135361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:58.135478 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:58.135606 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:58.135764 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 10:07:58.135883 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260808 10:07:58.137825 8429 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:07:58.137210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51440 (local address 127.2.30.129:40611)
0808 10:07:58.137263 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:58.137265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:58.137386 (+ 121us) server_negotiation.cc:407] Connection header received
0808 10:07:58.137442 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:58.137443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:58.137451 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:58.137491 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:58.137764 (+ 273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51440: BlockingRecv error: recv got EOF from 127.0.0.1:51440 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260808 10:07:58.138015 2170 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786183651681294-2170-0/krb5kdc/krb5cc' of type FILE
Aug 08 10:07:58 dist-test-slave-w206 krb5kdc[7898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:58.139221 2170 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 10:07:58 dist-test-slave-w206 krb5kdc[7898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:58.140194 2170 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 10:07:58 dist-test-slave-w206 krb5kdc[7898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.129@KRBTEST.COM
I20260808 10:07:58.148352 8429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:58.140786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51448 (local address 127.2.30.129:40611)
0808 10:07:58.140854 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:58.140856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:58.140932 (+ 76us) server_negotiation.cc:407] Connection header received
0808 10:07:58.141021 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:58.141022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:58.141033 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:58.141106 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:58.141575 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:58.142106 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:58.142829 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:58.143088 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:58.143166 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:58.145519 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:58.145521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:58.145521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:58.145529 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:58.146921 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:58.147449 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:58.147450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:58.147450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:58.147522 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:58.147728 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:58.147729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:58.147730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:58.147842 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:58.147923 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:58.148068 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 10:07:58.148194 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:07:58.149099 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 8008
I20260808 10:07:58.159159 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 8147
I20260808 10:07:58.165704 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 8287
I20260808 10:07:58.172360 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 7911
2026-08-08T10:07:58Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1951 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:07:58 dist-test-slave-w206 krb5kdc[8442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:07:58 dist-test-slave-w206 krb5kdc[8442](info): set up 2 sockets
Aug 08 10:07:58 dist-test-slave-w206 krb5kdc[8442](info): commencing operation
krb5kdc: starting...
W20260808 10:07:59.264377 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:07:59 dist-test-slave-w206 krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:07:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:07:59Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.190.keytab.
Entry for principal oryx/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.190.keytab.
I20260808 10:07:59.398401 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:41065
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.190.keytab
--principal=oryx/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:41065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:59.482712 8455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:59.482971 8455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:59.483040 8455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:59.484241 8455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:07:59.484318 8455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:59.484340 8455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:59.484361 8455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:07:59.484393 8455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:07:59.485882 8455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:41065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.190.keytab
--principal=oryx/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:41065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:59.486172 8455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:59.486415 8455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:59.491413 8461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:59.491451 8460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:59.491483 8455 server_base.cc:1034] running on GCE node
W20260808 10:07:59.491451 8463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:59.491899 8455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:59.492172 8455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:59.493359 8455 hybrid_clock.cc:648] HybridClock initialized: now 1786183679493326 us; error 44 us; skew 500 ppm
Aug 08 10:07:59 dist-test-slave-w206 krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183679, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:59.496130 8455 init.cc:377] Logged in from keytab as oryx/127.2.30.190@KRBTEST.COM (short username oryx)
I20260808 10:07:59.496582 8455 webserver.cc:460] Webserver started at http://127.2.30.190:44341/ using document root <none> and password file <none>
I20260808 10:07:59.496821 8455 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:59.496867 8455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:59.497041 8455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:59.498137 8455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "c974e52359c14e1aaf0c36f072c48dcc"
format_stamp: "Formatted at 2026-08-08 10:07:59 on dist-test-slave-w206"
server_key: "d9fe7010b0b30328503d95e7f3933a4d"
server_key_iv: "627901983830467db2dc66a9fc1a7c19"
server_key_version: "encryptionkey@0"
I20260808 10:07:59.498517 8455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "c974e52359c14e1aaf0c36f072c48dcc"
format_stamp: "Formatted at 2026-08-08 10:07:59 on dist-test-slave-w206"
server_key: "d9fe7010b0b30328503d95e7f3933a4d"
server_key_iv: "627901983830467db2dc66a9fc1a7c19"
server_key_version: "encryptionkey@0"
I20260808 10:07:59.500051 8455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:07:59.500933 8471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:59.501152 8455 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:59.501268 8455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "c974e52359c14e1aaf0c36f072c48dcc"
format_stamp: "Formatted at 2026-08-08 10:07:59 on dist-test-slave-w206"
server_key: "d9fe7010b0b30328503d95e7f3933a4d"
server_key_iv: "627901983830467db2dc66a9fc1a7c19"
server_key_version: "encryptionkey@0"
I20260808 10:07:59.501361 8455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:59.512800 8455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:59.515275 8455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:59.515457 8455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:59.519726 8455 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:41065
I20260808 10:07:59.519795 8524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:41065 every 8 connection(s)
I20260808 10:07:59.520218 8455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:07:59.521286 8525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:07:59.523795 8525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc: Bootstrap starting.
I20260808 10:07:59.524410 8525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc: Neither blocks nor log segments found. Creating new log.
I20260808 10:07:59.524593 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 8455
I20260808 10:07:59.524749 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:07:59.524771 8525 log.cc:824] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc: Log is configured to *not* fsync() on all Append() calls
I20260808 10:07:59.524972 2170 external_mini_cluster.cc:1428] Setting key f3d45a3a9a9929027a17bfcdd9b91067
I20260808 10:07:59.525722 8525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc: No bootstrap required, opened a new log
I20260808 10:07:59.528134 8525 raft_consensus.cc:348] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } }
I20260808 10:07:59.528285 8525 raft_consensus.cc:374] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:07:59.528335 8525 raft_consensus.cc:729] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c974e52359c14e1aaf0c36f072c48dcc, State: Initialized, Role: FOLLOWER
I20260808 10:07:59.528470 8525 consensus_queue.cc:260] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } }
I20260808 10:07:59.528553 8525 raft_consensus.cc:388] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:07:59.528577 8525 raft_consensus.cc:482] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:07:59.528644 8525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:07:59.529263 8525 raft_consensus.cc:504] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } }
I20260808 10:07:59.529388 8525 leader_election.cc:302] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c974e52359c14e1aaf0c36f072c48dcc; no voters:
I20260808 10:07:59.529585 8525 leader_election.cc:288] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:07:59.529639 8530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:07:59.529824 8530 raft_consensus.cc:686] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [term 1 LEADER]: Becoming Leader. State: Replica: c974e52359c14e1aaf0c36f072c48dcc, State: Running, Role: LEADER
I20260808 10:07:59.529862 8525 sys_catalog.cc:564] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:07:59.529932 8530 consensus_queue.cc:237] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } }
I20260808 10:07:59.530272 8530 sys_catalog.cc:455] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c974e52359c14e1aaf0c36f072c48dcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } }
I20260808 10:07:59.530347 8530 sys_catalog.cc:458] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:59.530515 8530 sys_catalog.cc:455] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c974e52359c14e1aaf0c36f072c48dcc. Latest consensus state: current_term: 1 leader_uuid: "c974e52359c14e1aaf0c36f072c48dcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } }
I20260808 10:07:59.530586 8530 sys_catalog.cc:458] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: This master's current role is: LEADER
I20260808 10:07:59.530746 8536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:07:59.531314 8536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 10:07:59 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM
I20260808 10:07:59.533159 8536 catalog_manager.cc:1347] Generated new cluster ID: 8987d372a5fd4b5aa3c86650e67245ab
I20260808 10:07:59.533209 8536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:07:59.536449 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:59.525606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59642 (local address 127.2.30.190:41065)
0808 10:07:59.525832 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:59.525841 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:59.525875 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:07:59.526315 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:59.526322 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:59.526372 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:59.526526 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:59.527240 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:59.528118 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:59.529042 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:59.529621 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:59.529792 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:59.532953 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:59.532957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:59.532960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:59.532977 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:59.534822 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:59.535296 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:59.535301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:59.535302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:59.535389 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:59.535612 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:59.535613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:59.535614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:59.535856 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:59.536035 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:59.536116 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 10:07:59.536272 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":102,"threads_started":1}
I20260808 10:07:59.543256 8536 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:07:59.544020 8536 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:07:59.549852 8536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc: Generated new TSK 0
I20260808 10:07:59.550055 8536 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.129.keytab.
Entry for principal oryx/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.129.keytab.
I20260808 10:07:59.586305 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41065
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.129.keytab
--principal=oryx/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:59.672009 8552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:59.672238 8552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:59.672266 8552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:59.673478 8552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:59.673533 8552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:59.673563 8552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:07:59.675210 8552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.129.keytab
--principal=oryx/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:59.675498 8552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:59.675766 8552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:59.681044 8557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:59.681044 8560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:59.681120 8558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:59.681351 8552 server_base.cc:1034] running on GCE node
I20260808 10:07:59.681550 8552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:59.681809 8552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:59.683038 8552 hybrid_clock.cc:648] HybridClock initialized: now 1786183679682963 us; error 31 us; skew 500 ppm
Aug 08 10:07:59 dist-test-slave-w206 krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183679, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:59.685876 8552 init.cc:377] Logged in from keytab as oryx/127.2.30.129@KRBTEST.COM (short username oryx)
I20260808 10:07:59.686337 8552 webserver.cc:460] Webserver started at http://127.2.30.129:35427/ using document root <none> and password file <none>
I20260808 10:07:59.686601 8552 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:59.686646 8552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:59.686779 8552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:59.687968 8552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "74d1601b5a52457a951fafd40b699521"
format_stamp: "Formatted at 2026-08-08 10:07:59 on dist-test-slave-w206"
server_key: "3b81ae3287a395ba70001f260317b016"
server_key_iv: "a0f015c668fdc8783c82d18ef10b65d6"
server_key_version: "encryptionkey@0"
I20260808 10:07:59.688350 8552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "74d1601b5a52457a951fafd40b699521"
format_stamp: "Formatted at 2026-08-08 10:07:59 on dist-test-slave-w206"
server_key: "3b81ae3287a395ba70001f260317b016"
server_key_iv: "a0f015c668fdc8783c82d18ef10b65d6"
server_key_version: "encryptionkey@0"
I20260808 10:07:59.689764 8552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:07:59.690656 8567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:59.690884 8552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:59.690999 8552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "74d1601b5a52457a951fafd40b699521"
format_stamp: "Formatted at 2026-08-08 10:07:59 on dist-test-slave-w206"
server_key: "3b81ae3287a395ba70001f260317b016"
server_key_iv: "a0f015c668fdc8783c82d18ef10b65d6"
server_key_version: "encryptionkey@0"
I20260808 10:07:59.691082 8552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:59.704532 8552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:59.707077 8552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:59.707335 8552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:59.707630 8552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:59.708009 8552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:59.708070 8552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:59.708109 8552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:59.708141 8552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:59.714677 8552 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:44221
I20260808 10:07:59.714727 8684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:44221 every 8 connection(s)
I20260808 10:07:59.715171 8552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
Aug 08 10:07:59 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183679, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM
I20260808 10:07:59.722431 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 8552
I20260808 10:07:59.722563 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:07:59.722795 2170 external_mini_cluster.cc:1428] Setting key 11ab8418ad89bf905a2a350c293d9a3c
I20260808 10:07:59.726109 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:59.716084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:47803 (local address 127.2.30.190:41065)
0808 10:07:59.716196 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:59.716198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:59.716368 (+ 170us) server_negotiation.cc:407] Connection header received
0808 10:07:59.716840 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:59.716841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:59.716853 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:59.716913 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:59.717924 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:59.718474 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:59.719543 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:59.719787 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:59.719853 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:59.722500 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:59.722502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:59.722502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:59.722517 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:59.724360 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:59.724805 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:59.724806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:59.724806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:59.724870 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:59.725145 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:59.725146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:59.725146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:59.725316 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:59.725395 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:59.725869 (+ 474us) server_negotiation.cc:299] Negotiation successful
0808 10:07:59.725979 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 10:07:59.726189 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:59.716000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.129:47803)
0808 10:07:59.716230 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:59.716270 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:59.716750 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:59.716991 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:59.716994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:59.717637 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:59.717832 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:59.717836 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:59.718560 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:59.718561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:59.719459 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:59.719461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:59.719914 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:59.719916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:59.720037 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:59.720700 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:59.720709 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:59.722373 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:59.724458 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:59.724467 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:59.724470 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:59.724713 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:59.724945 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:59.724946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:59.724946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:59.725066 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:59.725411 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:59.725421 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:59.725710 (+ 289us) client_negotiation.cc:769] Sending connection context
0808 10:07:59.725823 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 10:07:59.725972 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":72,"threads_started":1}
I20260808 10:07:59.726747 8686 heartbeater.cc:347] Connected to a master server at 127.2.30.190:41065
I20260808 10:07:59.726850 8686 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:59.727092 8686 heartbeater.cc:511] Master 127.2.30.190:41065 requested a full tablet report, sending...
I20260808 10:07:59.727753 8488 ts_manager.cc:194] Registered new tserver with Master: 74d1601b5a52457a951fafd40b699521 (127.2.30.129:44221)
I20260808 10:07:59.728777 8488 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:47803
WARNING: no policy specified for oryx/127.2.30.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.130.keytab.
Entry for principal oryx/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.130.keytab.
I20260808 10:07:59.751847 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41065
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.130.keytab
--principal=oryx/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:07:59.836284 8691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:07:59.836498 8691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:07:59.836525 8691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:07:59.837644 8691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:07:59.837684 8691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:07:59.837711 8691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:07:59.839174 8691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.130.keytab
--principal=oryx/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:07:59.839389 8691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:07:59.839588 8691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:07:59.844817 8697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:59.844828 8696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:07:59.844830 8699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:07:59.845055 8691 server_base.cc:1034] running on GCE node
I20260808 10:07:59.845321 8691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:07:59.845542 8691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:07:59.846700 8691 hybrid_clock.cc:648] HybridClock initialized: now 1786183679846677 us; error 39 us; skew 500 ppm
Aug 08 10:07:59 dist-test-slave-w206 krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183679, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:07:59.849521 8691 init.cc:377] Logged in from keytab as oryx/127.2.30.130@KRBTEST.COM (short username oryx)
I20260808 10:07:59.850003 8691 webserver.cc:460] Webserver started at http://127.2.30.130:35333/ using document root <none> and password file <none>
I20260808 10:07:59.850270 8691 fs_manager.cc:359] Metadata directory not provided
I20260808 10:07:59.850317 8691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:07:59.850471 8691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:07:59.851617 8691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "a116392f87ac403ab0d65007de2db5cf"
format_stamp: "Formatted at 2026-08-08 10:07:59 on dist-test-slave-w206"
server_key: "6209dec25d496b77366a75033e3f7539"
server_key_iv: "e82775733cdd6495fe92dcc1458a7b1b"
server_key_version: "encryptionkey@0"
I20260808 10:07:59.851987 8691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "a116392f87ac403ab0d65007de2db5cf"
format_stamp: "Formatted at 2026-08-08 10:07:59 on dist-test-slave-w206"
server_key: "6209dec25d496b77366a75033e3f7539"
server_key_iv: "e82775733cdd6495fe92dcc1458a7b1b"
server_key_version: "encryptionkey@0"
I20260808 10:07:59.853459 8691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:07:59.854269 8706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:59.854528 8691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:07:59.854636 8691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "a116392f87ac403ab0d65007de2db5cf"
format_stamp: "Formatted at 2026-08-08 10:07:59 on dist-test-slave-w206"
server_key: "6209dec25d496b77366a75033e3f7539"
server_key_iv: "e82775733cdd6495fe92dcc1458a7b1b"
server_key_version: "encryptionkey@0"
I20260808 10:07:59.854712 8691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:07:59.873363 8691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:07:59.875535 8691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:07:59.875689 8691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:07:59.875959 8691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:07:59.876287 8691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:07:59.876343 8691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:59.876385 8691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:07:59.876413 8691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:07:59.882459 8691 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:36601
I20260808 10:07:59.882520 8822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:36601 every 8 connection(s)
I20260808 10:07:59.882843 8691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:07:59.887816 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 8691
I20260808 10:07:59.887950 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:07:59.888160 2170 external_mini_cluster.cc:1428] Setting key 4823f4e87763415d1c405f2914155f13
Aug 08 10:07:59 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183679, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM
I20260808 10:07:59.893940 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:59.883592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:46293 (local address 127.2.30.190:41065)
0808 10:07:59.883659 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 10:07:59.883661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:07:59.883848 (+ 187us) server_negotiation.cc:407] Connection header received
0808 10:07:59.884442 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:07:59.884444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:07:59.884458 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:07:59.884518 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:07:59.885539 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:59.886100 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:59.887237 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:07:59.887442 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:07:59.887532 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:59.890371 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:07:59.890372 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:07:59.890372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:07:59.890381 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:07:59.892252 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:59.892786 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:59.892788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:59.892788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:59.892857 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:07:59.893118 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:07:59.893119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:07:59.893120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:07:59.893273 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:07:59.893343 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 10:07:59.893707 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 10:07:59.893807 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:07:59.894058 8825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:07:59.883530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.130:46293)
0808 10:07:59.883704 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 10:07:59.883758 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:07:59.884327 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:07:59.884586 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:07:59.884591 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:07:59.885217 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:07:59.885418 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:59.885423 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:59.886204 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:59.886205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:59.887138 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:07:59.887140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:07:59.887544 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:07:59.887545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:07:59.887681 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:07:59.888302 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:07:59.888310 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:07:59.890144 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:07:59.892363 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:59.892370 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:59.892372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:59.892697 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:59.892930 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:07:59.892931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:07:59.892931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:07:59.893052 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:07:59.893366 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:07:59.893371 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:07:59.893554 (+ 183us) client_negotiation.cc:769] Sending connection context
0808 10:07:59.893648 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 10:07:59.893781 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":69,"threads_started":1}
I20260808 10:07:59.894909 8823 heartbeater.cc:347] Connected to a master server at 127.2.30.190:41065
I20260808 10:07:59.895048 8823 heartbeater.cc:464] Registering TS with master...
I20260808 10:07:59.895320 8823 heartbeater.cc:511] Master 127.2.30.190:41065 requested a full tablet report, sending...
I20260808 10:07:59.895812 8488 ts_manager.cc:194] Registered new tserver with Master: a116392f87ac403ab0d65007de2db5cf (127.2.30.130:36601)
I20260808 10:07:59.896215 8488 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:46293
WARNING: no policy specified for oryx/127.2.30.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.131.keytab.
Entry for principal oryx/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.131.keytab.
I20260808 10:07:59.918936 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41065
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.131.keytab
--principal=oryx/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:00.005895 8828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:00.006156 8828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:00.006198 8828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:00.007489 8828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:00.007560 8828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:00.007606 8828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:08:00.009174 8828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.131.keytab
--principal=oryx/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:41065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:00.009473 8828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:00.009729 8828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:00.015102 8836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:00.015079 8834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:00.015084 8833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:00.015442 8828 server_base.cc:1034] running on GCE node
I20260808 10:08:00.015674 8828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:00.015952 8828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:00.017128 8828 hybrid_clock.cc:648] HybridClock initialized: now 1786183680017111 us; error 30 us; skew 500 ppm
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:00.020318 8828 init.cc:377] Logged in from keytab as oryx/127.2.30.131@KRBTEST.COM (short username oryx)
I20260808 10:08:00.020802 8828 webserver.cc:460] Webserver started at http://127.2.30.131:42459/ using document root <none> and password file <none>
I20260808 10:08:00.021018 8828 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:00.021065 8828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:00.021179 8828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:00.022336 8828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "71379382edcc4184aa937dbb8098251e"
format_stamp: "Formatted at 2026-08-08 10:08:00 on dist-test-slave-w206"
server_key: "97c0ae2a0bd1b606e9fdf04f1bb98463"
server_key_iv: "942359ef67430f20a76e9da1cd7d1202"
server_key_version: "encryptionkey@0"
I20260808 10:08:00.022706 8828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "71379382edcc4184aa937dbb8098251e"
format_stamp: "Formatted at 2026-08-08 10:08:00 on dist-test-slave-w206"
server_key: "97c0ae2a0bd1b606e9fdf04f1bb98463"
server_key_iv: "942359ef67430f20a76e9da1cd7d1202"
server_key_version: "encryptionkey@0"
I20260808 10:08:00.025003 8828 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 10:08:00.026173 8843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:00.026365 8828 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:00.026477 8828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "71379382edcc4184aa937dbb8098251e"
format_stamp: "Formatted at 2026-08-08 10:08:00 on dist-test-slave-w206"
server_key: "97c0ae2a0bd1b606e9fdf04f1bb98463"
server_key_iv: "942359ef67430f20a76e9da1cd7d1202"
server_key_version: "encryptionkey@0"
I20260808 10:08:00.026571 8828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:00.040191 8828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:00.042711 8828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:00.042886 8828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:00.043183 8828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:00.043509 8828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:00.043565 8828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:00.043608 8828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:00.043632 8828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:00.050069 8828 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:33039
I20260808 10:08:00.050107 8961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:33039 every 8 connection(s)
I20260808 10:08:00.050490 8828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:08:00.055401 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 8828
I20260808 10:08:00.055528 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:08:00.055783 2170 external_mini_cluster.cc:1428] Setting key bdea840021fb9c2cc3d7da653193ae49
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM
I20260808 10:08:00.060655 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.051336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:49723 (local address 127.2.30.190:41065)
0808 10:08:00.051423 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.051425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.051609 (+ 184us) server_negotiation.cc:407] Connection header received
0808 10:08:00.052083 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.052084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.052094 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.052165 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.053126 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.053628 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.054712 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.054900 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.054955 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.057808 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.057809 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.057810 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.057817 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.059371 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.059810 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.059811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.059811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.059865 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.060095 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.060095 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.060096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.060203 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.060256 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.060511 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.060587 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 10:08:00.060883 8964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.051310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.131:49723)
0808 10:08:00.051469 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.051513 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.052000 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.052228 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.052231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.052832 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.053035 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.053040 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.053735 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.053736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.054625 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.054628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.055018 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.055019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.055144 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.055804 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.055815 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.057718 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.059474 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.059482 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.059484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.059731 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.059931 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.059931 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.059932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.060038 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.060267 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.060272 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.060396 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.060483 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.060611 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260808 10:08:00.061470 8962 heartbeater.cc:347] Connected to a master server at 127.2.30.190:41065
I20260808 10:08:00.061581 8962 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:00.061784 8962 heartbeater.cc:511] Master 127.2.30.190:41065 requested a full tablet report, sending...
I20260808 10:08:00.062198 8488 ts_manager.cc:194] Registered new tserver with Master: 71379382edcc4184aa937dbb8098251e (127.2.30.131:33039)
I20260808 10:08:00.062564 8488 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:49723
I20260808 10:08:00.067909 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.30.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.30.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.30.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.189.keytab.
Entry for principal oryx/127.2.30.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.189.keytab.
I20260808 10:08:00.098037 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.189:46359
--webserver_interface=127.2.30.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.189.keytab
--principal=oryx/127.2.30.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:41065,127.2.30.189:46359
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:00.183044 8967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:00.183281 8967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:00.183305 8967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:00.184567 8967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:08:00.184620 8967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:00.184634 8967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:00.184648 8967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:08:00.184661 8967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:08:00.186210 8967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:41065,127.2.30.189:46359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.189.keytab
--principal=oryx/127.2.30.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.189:46359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.30.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
W20260808 10:08:00.186323 8967 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.30.190:41065,127.2.30.189:46359'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 10:08:00.186519 8967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:00.186764 8967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:00.192199 8974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:00.192229 8973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:00.192294 8976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:00.192394 8967 server_base.cc:1034] running on GCE node
I20260808 10:08:00.192643 8967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:00.192998 8967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:00.194198 8967 hybrid_clock.cc:648] HybridClock initialized: now 1786183680194168 us; error 43 us; skew 500 ppm
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:00.197064 8967 init.cc:377] Logged in from keytab as oryx/127.2.30.189@KRBTEST.COM (short username oryx)
I20260808 10:08:00.197477 8967 webserver.cc:460] Webserver started at http://127.2.30.189:39807/ using document root <none> and password file <none>
I20260808 10:08:00.197688 8967 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:00.197731 8967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:00.197824 8967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:00.198872 8967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/data/instance:
uuid: "9f3602e8df67413eaf140373844707a5"
format_stamp: "Formatted at 2026-08-08 10:08:00 on dist-test-slave-w206"
server_key: "db5fea8030ecbdfa2982b5d1b150aee0"
server_key_iv: "c3fd23352884a1aa814f045beeeb80ef"
server_key_version: "encryptionkey@0"
I20260808 10:08:00.199261 8967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/wal/instance:
uuid: "9f3602e8df67413eaf140373844707a5"
format_stamp: "Formatted at 2026-08-08 10:08:00 on dist-test-slave-w206"
server_key: "db5fea8030ecbdfa2982b5d1b150aee0"
server_key_iv: "c3fd23352884a1aa814f045beeeb80ef"
server_key_version: "encryptionkey@0"
I20260808 10:08:00.200637 8967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:00.201387 8983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:00.201584 8967 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:00.201678 8967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/wal
uuid: "9f3602e8df67413eaf140373844707a5"
format_stamp: "Formatted at 2026-08-08 10:08:00 on dist-test-slave-w206"
server_key: "db5fea8030ecbdfa2982b5d1b150aee0"
server_key_iv: "c3fd23352884a1aa814f045beeeb80ef"
server_key_version: "encryptionkey@0"
I20260808 10:08:00.201730 8967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:00.218047 8967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:00.220635 8967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:00.220764 8967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:00.225062 8967 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.189:46359
I20260808 10:08:00.225127 9036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.189:46359 every 8 connection(s)
I20260808 10:08:00.225481 8967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/data/info.pb
I20260808 10:08:00.226869 9038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:00.228580 9038 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:08:00.234331 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 8967
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.189@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM
I20260808 10:08:00.245365 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.229563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59650 (local address 127.2.30.190:41065)
0808 10:08:00.229611 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.229613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.229830 (+ 217us) server_negotiation.cc:407] Connection header received
0808 10:08:00.230501 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.230503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.230515 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.230596 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.231564 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.232190 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.233755 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.234937 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.235064 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.241838 (+ 6774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.241840 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.241841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.241852 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.243515 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.244057 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.244058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.244059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.244122 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.244444 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.244445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.244446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.244670 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.244722 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.245143 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.245245 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:08:00.245838 9040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.229443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:59650)
0808 10:08:00.229680 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.229729 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.230320 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.230697 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.230701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.231091 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.231423 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.231429 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.232298 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.232299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.233648 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.233651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.238847 (+ 5196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.238849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.239083 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.239762 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.239774 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.241708 (+ 1934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.243616 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.243626 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.243629 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.243957 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.244199 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.244200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.244200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.244358 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.244779 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.244786 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.244968 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.245092 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.245274 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":153,"threads_started":1}
I20260808 10:08:00.246899 9038 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.30.189@KRBTEST.COM
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.189@KRBTEST.COM for oryx/127.2.30.189@KRBTEST.COM
I20260808 10:08:00.252379 9042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.240640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46180 (local address 127.2.30.189:46359)
0808 10:08:00.242610 (+ 1970us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.242617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.242645 (+ 28us) server_negotiation.cc:407] Connection header received
0808 10:08:00.242668 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.242671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.242689 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.242805 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.243519 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.244424 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.245232 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.245679 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.245753 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.248996 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.249006 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.249007 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.249015 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.251099 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.251507 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.251511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.251512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.251585 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.251765 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.251766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.251766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.251970 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.252119 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.252222 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.252291 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1928,"thread_start_us":119,"threads_started":1}
I20260808 10:08:00.254822 9040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.247461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:46359 (local address 127.0.0.1:46190)
0808 10:08:00.247504 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.247517 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.247568 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.248024 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.248025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.248206 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.248265 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.248266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.249062 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.249063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.249771 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.249774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.250167 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.250167 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.250233 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.250610 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.250614 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.252168 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.253989 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.253990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.253990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.254152 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.254364 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.254365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.254365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.254417 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.254596 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.254597 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.254632 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.254678 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.254757 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 10:08:00.254982 9043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.247702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46190 (local address 127.2.30.189:46359)
0808 10:08:00.247887 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.247889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.247897 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:08:00.247917 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.247917 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.247934 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.247994 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.248352 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.248958 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.249848 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.250079 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.250166 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.252253 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.252254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.252254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.252260 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.253918 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.254212 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.254213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.254213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.254259 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.254457 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.254458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.254458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.254545 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.254591 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.254699 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.254811 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":165,"thread_start_us":137,"threads_started":1}
I20260808 10:08:00.256428 9038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: Bootstrap starting.
I20260808 10:08:00.257097 9038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:00.257485 9038 log.cc:824] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:00.258579 9038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: No bootstrap required, opened a new log
I20260808 10:08:00.260288 9038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } }
I20260808 10:08:00.260424 9038 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:00.260448 9038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3602e8df67413eaf140373844707a5, State: Initialized, Role: FOLLOWER
I20260808 10:08:00.260608 9038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } }
I20260808 10:08:00.260896 9045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } } }
I20260808 10:08:00.260965 9038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:08:00.261027 9045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:08:00.262744 9057 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:08:00.262822 9057 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:08:00.264806 8967 master_runner.cc:428] Detected that this master 9f3602e8df67413eaf140373844707a5 is joining an existing cluster
I20260808 10:08:00.264874 8967 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.30.189:46359
I20260808 10:08:00.272976 9065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.265883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:46359 (local address 127.0.0.1:46204)
0808 10:08:00.266040 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.266050 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.266091 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.266251 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.266252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.266479 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.266541 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.266543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.267338 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.267338 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.267899 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.267900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.268287 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.268288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.268353 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.269151 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.269155 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.269584 (+ 429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.271700 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.271701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.271702 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.271973 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.272219 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.272220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.272220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.272286 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.272602 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.272602 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.272653 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.272725 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.272842 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260808 10:08:00.272984 9043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.266111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46204 (local address 127.2.30.189:46359)
0808 10:08:00.266150 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.266152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.266160 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:08:00.266180 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.266180 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.266187 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.266247 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.266617 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.267243 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.267981 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.268185 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.268271 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.269665 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.269666 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.269667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.269674 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.271601 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.272101 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.272103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.272103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.272166 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.272359 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.272360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.272360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.272517 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.272580 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.272761 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.272896 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":22}
I20260808 10:08:00.273945 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.266069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59662 (local address 127.2.30.190:41065)
0808 10:08:00.266165 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.266167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.266177 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:08:00.266206 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.266207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.266219 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.266284 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.266780 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.267638 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.268806 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.269022 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.269078 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.270239 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.270240 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.270241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.270248 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.272086 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.272448 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.272449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.272449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.272502 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.272764 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.272766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.272766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.272912 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.272959 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.273784 (+ 825us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.273870 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 10:08:00.274164 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.265833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:59662)
0808 10:08:00.265987 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.266000 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.266059 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.266335 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.266336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.266590 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.266659 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.266661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.267762 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.267763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.268712 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.268715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.269103 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.269104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.269176 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.269631 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.269637 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.270086 (+ 449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.272153 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.272153 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.272154 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.272370 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.272577 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.272578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.272578 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.272653 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.273012 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.273013 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.273618 (+ 605us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.273746 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.273860 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":74,"threads_started":1}
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183679, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.190@KRBTEST.COM for oryx/127.2.30.189@KRBTEST.COM
I20260808 10:08:00.288295 9043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.276959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46214 (local address 127.2.30.189:46359)
0808 10:08:00.277023 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.277025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.279262 (+ 2237us) server_negotiation.cc:407] Connection header received
0808 10:08:00.279371 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.279372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.279383 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.279443 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.280131 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.280737 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.282107 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.282386 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.282466 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.285148 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.285149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.285150 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.285156 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.286756 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.287391 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.287392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.287392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.287457 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.287734 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.287735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.287735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.287872 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.287934 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.288154 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.288224 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 10:08:00.288533 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.276856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:46359 (local address 127.0.0.1:46214)
0808 10:08:00.279109 (+ 2253us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.279174 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.279275 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.279472 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.279479 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.279933 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.280011 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.280014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.280832 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.280834 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.281992 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.281994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.282483 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.282485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.282597 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.283237 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.283243 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.285043 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.286860 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.286870 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.286873 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.287224 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.287545 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.287546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.287546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.287629 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.287948 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.287959 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.288026 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.288117 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.288262 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2212,"thread_start_us":132,"threads_started":1}
I20260808 10:08:00.289203 8488 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.30.189:46359: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9f3602e8df67413eaf140373844707a5"
member_type: NON_VOTER
last_known_addr {
host: "127.2.30.189"
port: 46359
}
attrs {
promote: true
}
}
dest_uuid: "c974e52359c14e1aaf0c36f072c48dcc"
cas_config_opid_index: -1
I20260808 10:08:00.289566 8488 consensus_queue.cc:237] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: NON_VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: true } }
I20260808 10:08:00.290153 9068 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 9f3602e8df67413eaf140373844707a5 (127.2.30.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: NON_VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: true } } }
I20260808 10:08:00.290541 9010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:00.291414 9068 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.30.189:46359
I20260808 10:08:00.291636 9010 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 FOLLOWER]: Refusing update from remote peer c974e52359c14e1aaf0c36f072c48dcc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 10:08:00.291790 8967 master.cc:537] Master@127.2.30.189:46359 shutting down...
I20260808 10:08:00.292028 9067 sys_catalog.cc:455] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c974e52359c14e1aaf0c36f072c48dcc" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: NON_VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: true } } }
I20260808 10:08:00.292102 9067 sys_catalog.cc:458] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:00.292514 9067 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: NON_VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:08:00.293635 8474 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 10:08:00.294602 8474 consensus_peers.cc:487] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc -> Peer 9f3602e8df67413eaf140373844707a5 (127.2.30.189:46359): Couldn't send request to peer 9f3602e8df67413eaf140373844707a5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 10:08:00.295954 8967 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:08:00.296092 8967 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:00.296152 8967 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: stopping tablet replica
I20260808 10:08:00.309576 8967 master.cc:552] Master@127.2.30.189:46359 shutdown complete.
I20260808 10:08:00.309664 8967 master_runner.cc:305] Clearing existing system tablet
I20260808 10:08:00.309820 8967 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:00.309893 8967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:00.310647 8967 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260808 10:08:00.311318 9072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:00.311533 8967 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:00.311625 8967 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:08:00.311666 8967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/wal
uuid: "9f3602e8df67413eaf140373844707a5"
format_stamp: "Formatted at 2026-08-08 10:08:00 on dist-test-slave-w206"
server_key: "db5fea8030ecbdfa2982b5d1b150aee0"
server_key_iv: "c3fd23352884a1aa814f045beeeb80ef"
server_key_version: "encryptionkey@0"
I20260808 10:08:00.312650 8967 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:00.314527 8967 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 10:08:00.314618 8967 log.cc:1196] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 10:08:00.314873 8967 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: Deleting consensus metadata
I20260808 10:08:00.315066 8967 master_runner.cc:315] Copying system tablet from 127.2.30.190:41065
I20260808 10:08:00.315804 8967 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: tablet copy: Beginning tablet copy session from remote peer at address 127.2.30.190:41065
I20260808 10:08:00.322788 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.316380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59664 (local address 127.2.30.190:41065)
0808 10:08:00.316457 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.316459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.316471 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:08:00.316578 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.316580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.316592 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.316654 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.317027 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.317588 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.318411 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.318637 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.318737 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.319869 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.319870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.319871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.319879 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.321468 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.321885 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.321887 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.321887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.321953 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.322192 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.322193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.322194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.322319 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.322405 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.322582 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.322682 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:08:00.322806 9078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.316248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:59664)
0808 10:08:00.316383 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.316399 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.316487 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.316671 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.316672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.316878 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.316936 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.316938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.317681 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.317683 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.318303 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.318305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.318740 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.318741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.318822 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.319400 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.319405 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.319739 (+ 334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.321584 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.321585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.321585 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.321789 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.322035 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.322037 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.322037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.322111 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.322395 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.322396 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.322451 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.322550 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.322674 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":87,"threads_started":1}
I20260808 10:08:00.323462 8509 tablet_copy_service.cc:140] P c974e52359c14e1aaf0c36f072c48dcc: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9f3602e8df67413eaf140373844707a5 ({username='oryx', principal='oryx/127.2.30.189@KRBTEST.COM'} at 127.0.0.1:59664)
I20260808 10:08:00.323581 8509 tablet_copy_service.cc:161] P c974e52359c14e1aaf0c36f072c48dcc: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9f3602e8df67413eaf140373844707a5 at {username='oryx', principal='oryx/127.2.30.189@KRBTEST.COM'} at 127.0.0.1:59664: session id = 9f3602e8df67413eaf140373844707a5-00000000000000000000000000000000
I20260808 10:08:00.324471 8509 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 10:08:00.325255 8967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 10:08:00.326889 8967 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: tablet copy: Starting download of 0 data blocks...
I20260808 10:08:00.327148 8967 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: tablet copy: Starting download of 1 WAL segments...
I20260808 10:08:00.328410 8967 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 10:08:00.329702 8509 tablet_copy_service.cc:342] P c974e52359c14e1aaf0c36f072c48dcc: Request end of tablet copy session 9f3602e8df67413eaf140373844707a5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.30.189@KRBTEST.COM'} at 127.0.0.1:59664
I20260808 10:08:00.329794 8509 tablet_copy_service.cc:433] P c974e52359c14e1aaf0c36f072c48dcc: ending tablet copy session 9f3602e8df67413eaf140373844707a5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9f3602e8df67413eaf140373844707a5
I20260808 10:08:00.332765 8967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:00.334122 9080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:00.334147 9079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:00.334314 9082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:00.335186 8967 server_base.cc:1034] running on GCE node
I20260808 10:08:00.335327 8967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:00.335505 8967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:00.336663 8967 hybrid_clock.cc:648] HybridClock initialized: now 1786183680336657 us; error 36 us; skew 500 ppm
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:00.338187 8967 init.cc:377] Logged in from keytab as oryx/127.2.30.189@KRBTEST.COM (short username oryx)
I20260808 10:08:00.338516 8967 webserver.cc:460] Webserver started at http://127.2.30.189:36361/ using document root <none> and password file <none>
I20260808 10:08:00.338647 8967 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:00.338706 8967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:00.339449 8967 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:00.340073 9089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:00.340241 8967 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:08:00.340302 8967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/wal
uuid: "9f3602e8df67413eaf140373844707a5"
format_stamp: "Formatted at 2026-08-08 10:08:00 on dist-test-slave-w206"
server_key: "db5fea8030ecbdfa2982b5d1b150aee0"
server_key_iv: "c3fd23352884a1aa814f045beeeb80ef"
server_key_version: "encryptionkey@0"
I20260808 10:08:00.340389 8967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:00.351363 8967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:00.353616 8967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:00.357290 8967 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.189:46359
I20260808 10:08:00.357318 9141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.189:46359 every 8 connection(s)
I20260808 10:08:00.357648 8967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-1/data/info.pb
I20260808 10:08:00.358696 9142 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:08:00.359289 9142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: Bootstrap starting.
I20260808 10:08:00.362453 9142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:08:00.362684 9142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: Bootstrap complete.
I20260808 10:08:00.362900 9142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: NON_VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: true } }
I20260808 10:08:00.363019 9142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9f3602e8df67413eaf140373844707a5, State: Initialized, Role: LEARNER
I20260808 10:08:00.363065 9142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: NON_VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: true } }
I20260808 10:08:00.363276 9142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:08:00.363257 9145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: NON_VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: true } } }
I20260808 10:08:00.363376 9145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: This master's current role is: LEARNER
I20260808 10:08:00.364980 9156 catalog_manager.cc:1259] Loaded cluster ID: 8987d372a5fd4b5aa3c86650e67245ab
I20260808 10:08:00.365039 9156 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: loading cluster ID for follower catalog manager: success
I20260808 10:08:00.365952 9156 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: acquiring CA information for follower catalog manager: success
I20260808 10:08:00.366233 9156 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:08:00.412191 9157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.405225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46248 (local address 127.2.30.189:46359)
0808 10:08:00.405403 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.405405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.405413 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:08:00.405436 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.405436 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.405446 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.405525 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.406113 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.406836 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.407637 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.407849 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.407949 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.409261 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.409262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.409262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.409270 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.410812 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.411258 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.411259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.411260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.411333 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.411541 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.411541 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.411542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.411658 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.411779 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.411950 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.412042 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for oryx/127.2.30.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.30.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.30.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.188.keytab.
Entry for principal oryx/127.2.30.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.188.keytab.
I20260808 10:08:00.442909 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.188:39837
--webserver_interface=127.2.30.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.188.keytab
--principal=oryx/127.2.30.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:41065,127.2.30.189:46359,127.2.30.188:39837
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:00.527958 9160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:00.528169 9160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:00.528191 9160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:00.529517 9160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:08:00.529562 9160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:00.529577 9160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:00.529590 9160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:08:00.529630 9160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:08:00.531778 9160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:41065,127.2.30.189:46359,127.2.30.188:39837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.188.keytab
--principal=oryx/127.2.30.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.188:39837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.30.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:00.532065 9160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:00.532346 9160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:00.537424 9165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:00.537410 9166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:00.537391 9168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:00.537688 9160 server_base.cc:1034] running on GCE node
I20260808 10:08:00.537938 9160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:00.538259 9160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:00.539405 9160 hybrid_clock.cc:648] HybridClock initialized: now 1786183680539396 us; error 29 us; skew 500 ppm
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:00.542285 9160 init.cc:377] Logged in from keytab as oryx/127.2.30.188@KRBTEST.COM (short username oryx)
I20260808 10:08:00.542824 9160 webserver.cc:460] Webserver started at http://127.2.30.188:44555/ using document root <none> and password file <none>
I20260808 10:08:00.543138 9160 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:00.543295 9160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:00.543473 9160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:00.544603 9160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/data/instance:
uuid: "6b52febb05ad477dac2b5d72e786208c"
format_stamp: "Formatted at 2026-08-08 10:08:00 on dist-test-slave-w206"
server_key: "42be1ea18d8a756cc21401d09b4b8b6b"
server_key_iv: "07250eec200e3ac862d89f201c1975d7"
server_key_version: "encryptionkey@0"
I20260808 10:08:00.544992 9160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/wal/instance:
uuid: "6b52febb05ad477dac2b5d72e786208c"
format_stamp: "Formatted at 2026-08-08 10:08:00 on dist-test-slave-w206"
server_key: "42be1ea18d8a756cc21401d09b4b8b6b"
server_key_iv: "07250eec200e3ac862d89f201c1975d7"
server_key_version: "encryptionkey@0"
I20260808 10:08:00.546545 9160 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:08:00.547524 9175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:00.547784 9160 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:08:00.547901 9160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/wal
uuid: "6b52febb05ad477dac2b5d72e786208c"
format_stamp: "Formatted at 2026-08-08 10:08:00 on dist-test-slave-w206"
server_key: "42be1ea18d8a756cc21401d09b4b8b6b"
server_key_iv: "07250eec200e3ac862d89f201c1975d7"
server_key_version: "encryptionkey@0"
I20260808 10:08:00.547996 9160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:00.567931 9160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:00.570495 9160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:00.570699 9160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:00.575292 9160 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.188:39837
I20260808 10:08:00.575397 9229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.188:39837 every 8 connection(s)
I20260808 10:08:00.575758 9160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/data/info.pb
I20260808 10:08:00.576732 9230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:00.578318 9230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:08:00.579130 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 9160
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.188@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.30.188@KRBTEST.COM
I20260808 10:08:00.589573 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.579245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59680 (local address 127.2.30.190:41065)
0808 10:08:00.579309 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.579311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.579536 (+ 225us) server_negotiation.cc:407] Connection header received
0808 10:08:00.580149 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.580150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.580172 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.580228 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.580977 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.581838 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.583098 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.583471 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.583554 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.586285 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.586286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.586287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.586295 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.588051 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.588528 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.588530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.588530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.588594 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.588865 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.588866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.588866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.588977 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.589032 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.589386 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.589475 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 10:08:00.589721 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.579102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:59680)
0808 10:08:00.579377 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.579429 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.580049 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.580271 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.580274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.580635 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.580882 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.580888 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.581938 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.581939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.583021 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.583024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.583566 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.583567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.583737 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.584467 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.584478 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.586185 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.588141 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.588146 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.588148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.588438 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.588668 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.588668 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.588668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.588769 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.589044 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.589051 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.589208 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.589342 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.589503 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":38,"thread_start_us":90,"threads_started":1}
I20260808 10:08:00.590312 9230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:08:00.590415 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.579765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56894 (local address 127.2.30.188:39837)
0808 10:08:00.579975 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.579980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.580009 (+ 29us) server_negotiation.cc:407] Connection header received
0808 10:08:00.580075 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.580080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.580116 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.580213 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.580986 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.582144 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.583166 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.583689 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.583763 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.586959 (+ 3196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.586967 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.586967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.586999 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.588752 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.589128 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.589130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.589131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.589190 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.589407 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.589408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.589408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.589581 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.589789 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.590174 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.590253 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":169,"thread_start_us":111,"threads_started":1}
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.188@KRBTEST.COM for oryx/127.2.30.189@KRBTEST.COM
I20260808 10:08:00.597218 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.590655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:46359 (local address 127.0.0.1:46252)
0808 10:08:00.590848 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.590861 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.590940 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.591137 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.591138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.591284 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.591321 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.591322 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.591996 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.591997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.592522 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.592523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.592923 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.592924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.592985 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.593414 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.593418 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.594600 (+ 1182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.596145 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.596146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.596147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.596332 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.596558 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.596559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.596559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.596624 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.596889 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.596890 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.596934 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.597003 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.597110 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260808 10:08:00.597218 9157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.590747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46252 (local address 127.2.30.189:46359)
0808 10:08:00.590852 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.590854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.590924 (+ 70us) server_negotiation.cc:407] Connection header received
0808 10:08:00.591063 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.591064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.591076 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.591135 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.591408 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.591912 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.592608 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.592836 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.592924 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.594699 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.594700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.594701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.594711 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.596045 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.596423 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.596425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.596425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.596488 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.596700 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.596701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.596702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.596819 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.596897 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.597049 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.597132 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 10:08:00.597780 9230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.188@KRBTEST.COM for oryx/127.2.30.188@KRBTEST.COM
I20260808 10:08:00.604646 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.598069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:39837 (local address 127.0.0.1:56904)
0808 10:08:00.598104 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.598129 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.598173 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.598293 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.598294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.598426 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.598451 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.598453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.599267 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.599269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.599730 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.599732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.600129 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.600130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.600176 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.600512 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.600516 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.601670 (+ 1154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.603073 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.603073 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.603074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.603241 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.603679 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.603680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.603680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.603750 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.604273 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.604273 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.604312 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.604398 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.604524 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 10:08:00.604707 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.598100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56904 (local address 127.2.30.188:39837)
0808 10:08:00.598153 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.598155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.598164 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:08:00.598229 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.598230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.598239 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.598293 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.598532 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.599166 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.599829 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.600058 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.600133 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.601741 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.601742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.601742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.601748 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.602948 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.603466 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.603467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.603468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.603520 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.603826 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.603827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.603827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.603935 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.604052 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.604421 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.604581 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":35}
I20260808 10:08:00.606204 9230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: Bootstrap starting.
I20260808 10:08:00.607228 9230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:00.607530 9230 log.cc:824] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:00.608311 9230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: No bootstrap required, opened a new log
I20260808 10:08:00.609843 9230 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } }
I20260808 10:08:00.609972 9230 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:00.609994 9230 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b52febb05ad477dac2b5d72e786208c, State: Initialized, Role: FOLLOWER
I20260808 10:08:00.610144 9230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } }
I20260808 10:08:00.610514 9230 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:08:00.610467 9235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } } }
I20260808 10:08:00.610591 9235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:08:00.612375 9246 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:08:00.612440 9246 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:08:00.614846 9160 master_runner.cc:428] Detected that this master 6b52febb05ad477dac2b5d72e786208c is joining an existing cluster
I20260808 10:08:00.614907 9160 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.30.188:39837
I20260808 10:08:00.623636 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.616185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56910 (local address 127.2.30.188:39837)
0808 10:08:00.616226 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.616228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.616550 (+ 322us) server_negotiation.cc:407] Connection header received
0808 10:08:00.616579 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.616579 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.616589 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.616649 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.617048 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.617782 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.618950 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.619279 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.619322 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.620528 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.620529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.620529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.620535 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.622166 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.622578 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.622579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.622580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.622635 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.622905 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.622907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.622907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.623104 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.623172 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.623392 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.623521 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":22}
I20260808 10:08:00.623756 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.616010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:39837 (local address 127.0.0.1:56910)
0808 10:08:00.616474 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.616485 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.616537 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.616688 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.616688 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.616901 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.616964 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.616967 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.617873 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.617873 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.618856 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.618858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.619407 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.619408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.619465 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.619919 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.619926 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.620420 (+ 494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.622259 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.622260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.622261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.622497 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.622705 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.622705 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.622706 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.622777 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.623179 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.623180 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.623246 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.623338 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.623464 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":46,"thread_start_us":84,"threads_started":1}
I20260808 10:08:00.631775 9235 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:08:00.631943 9235 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } }
I20260808 10:08:00.632272 9235 leader_election.cc:288] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c974e52359c14e1aaf0c36f072c48dcc (127.2.30.190:41065), 9f3602e8df67413eaf140373844707a5 (127.2.30.189:46359)
I20260808 10:08:00.632596 9116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b52febb05ad477dac2b5d72e786208c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f3602e8df67413eaf140373844707a5" is_pre_election: true
I20260808 10:08:00.632597 8499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b52febb05ad477dac2b5d72e786208c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c974e52359c14e1aaf0c36f072c48dcc" is_pre_election: true
I20260808 10:08:00.632740 9116 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 LEARNER]: Handling vote request from an unknown peer 6b52febb05ad477dac2b5d72e786208c
I20260808 10:08:00.632803 9116 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 6b52febb05ad477dac2b5d72e786208c for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260808 10:08:00.633026 9178 leader_election.cc:302] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b52febb05ad477dac2b5d72e786208c; no voters: 9f3602e8df67413eaf140373844707a5, c974e52359c14e1aaf0c36f072c48dcc
I20260808 10:08:00.633173 9235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:00.634109 9235 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 10:08:00.642231 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 8552
I20260808 10:08:00.649902 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:44221
--local_ip_for_outbound_sockets=127.2.30.129
--tserver_master_addrs=127.2.30.190:41065
--webserver_port=35427
--webserver_interface=127.2.30.129
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.129.keytab
--principal=oryx/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.30.190:41065,127.2.30.189:46359
--tserver_master_addrs=127.2.30.190:41065,127.2.30.189:46359,127.2.30.188:39837 with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:08:00.655846 9157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.616401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46256 (local address 127.2.30.189:46359)
0808 10:08:00.616442 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.616444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.616667 (+ 223us) server_negotiation.cc:407] Connection header received
0808 10:08:00.616776 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.616777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.616789 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.616819 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.617247 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.617962 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.618966 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.619292 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.619334 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.652474 (+ 33140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.652478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.652479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.652489 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.654368 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.654928 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.654930 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.654930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.655017 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.655236 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.655237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.655237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.655383 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.655426 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.655642 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.655735 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:08:00.656073 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.616035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59694 (local address 127.2.30.190:41065)
0808 10:08:00.616199 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.616201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.616214 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:08:00.616339 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.616340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.616353 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.616388 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.616888 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.617734 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.618952 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.619295 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.619334 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.652593 (+ 33259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.652594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.652595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.652605 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.654511 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.655093 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.655094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.655094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.655160 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.655389 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.655389 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.655390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.655551 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.655598 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.655849 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.655954 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260808 10:08:00.656232 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.616343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:46359 (local address 127.0.0.1:46256)
0808 10:08:00.616615 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.616623 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.616703 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.616887 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.616888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.617090 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.617150 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.617152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.618046 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.618047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.618878 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.618880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.619417 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.619419 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.619487 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.651588 (+ 32101us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.651597 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.652312 (+ 715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.654480 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.654481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.654482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.654823 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.655093 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.655094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.655095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.655152 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.655471 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.655472 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.655533 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.655991 (+ 458us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.656109 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":3,"thread_start_us":69,"threads_started":1}
I20260808 10:08:00.656510 9253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.615983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:59694)
0808 10:08:00.616151 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.616163 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.616241 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.616461 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.616462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.616706 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.616797 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.616800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.617825 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.617826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.618847 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.618850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.619419 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.619420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.619505 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.651998 (+ 32493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.652004 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.652470 (+ 466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.654609 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.654610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.654611 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.654959 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.655241 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.655242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.655243 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.655315 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.655649 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.655650 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.655702 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.656201 (+ 499us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.656354 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":42,"thread_start_us":104,"threads_started":1}
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183679, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.190@KRBTEST.COM for oryx/127.2.30.188@KRBTEST.COM
I20260808 10:08:00.668437 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.659326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:39837 (local address 127.0.0.1:56926)
0808 10:08:00.659379 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.659389 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.659427 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.659732 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.659733 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.659922 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.659972 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.659974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.661033 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.661035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.661855 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.661858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.662407 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.662408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.662478 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.662951 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.662956 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.664939 (+ 1983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.667261 (+ 2322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.667262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.667263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.667499 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.667707 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.667707 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.667708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.667782 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.668078 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.668079 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.668129 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.668195 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.668316 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 10:08:00.668861 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.659508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56926 (local address 127.2.30.188:39837)
0808 10:08:00.659576 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.659578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.659589 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:08:00.659629 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.659630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.659644 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.659728 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.660059 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.660919 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.661951 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.662316 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.662403 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.665061 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.665063 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.665063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.665071 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.667157 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.667581 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.667582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.667583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.667638 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.667853 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.667854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.667854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.667993 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.668052 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.668662 (+ 610us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.668756 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 10:08:00.669389 8488 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.30.188:39837: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6b52febb05ad477dac2b5d72e786208c"
member_type: NON_VOTER
last_known_addr {
host: "127.2.30.188"
port: 39837
}
attrs {
promote: true
}
}
dest_uuid: "c974e52359c14e1aaf0c36f072c48dcc"
cas_config_opid_index: 5
I20260808 10:08:00.669656 8488 consensus_queue.cc:237] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: NON_VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: true } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } }
I20260808 10:08:00.670233 9067 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6b52febb05ad477dac2b5d72e786208c (127.2.30.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: NON_VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: true } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } } }
I20260808 10:08:00.671311 9067 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.30.188:39837
I20260808 10:08:00.671422 9257 sys_catalog.cc:455] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c974e52359c14e1aaf0c36f072c48dcc" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: NON_VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: true } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } } }
I20260808 10:08:00.671667 9257 sys_catalog.cc:458] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:00.671789 9160 master.cc:537] Master@127.2.30.188:39837 shutting down...
I20260808 10:08:00.671883 9203 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 1 FOLLOWER]: Refusing update from remote peer c974e52359c14e1aaf0c36f072c48dcc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 10:08:00.672191 9257 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:08:00.672999 8472 consensus_peers.cc:487] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc -> Peer 6b52febb05ad477dac2b5d72e786208c (127.2.30.188:39837): Couldn't send request to peer 6b52febb05ad477dac2b5d72e786208c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 10:08:00.675024 9160 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:08:00.675104 9160 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:00.675144 9160 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: stopping tablet replica
I20260808 10:08:00.676617 9157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.670546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46264 (local address 127.2.30.189:46359)
0808 10:08:00.670591 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.670593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.672232 (+ 1639us) server_negotiation.cc:407] Connection header received
0808 10:08:00.672346 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.672347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.672375 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.672418 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:00.672622 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.673487 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.675356 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.676090 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.676143 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.676219 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.676476 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.676524 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:08:00.676869 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.670372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:46359 (local address 127.0.0.1:46264)
0808 10:08:00.672176 (+ 1804us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.672188 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.672265 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.672495 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.672496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.672497 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:00.672543 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.672544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.673736 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.673737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.675234 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.675238 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.676287 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.676288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.676361 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.676383 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.676749 (+ 366us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.676796 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1788}
I20260808 10:08:00.677673 9116 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 LEARNER]: Refusing update from remote peer c974e52359c14e1aaf0c36f072c48dcc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 10:08:00.677876 9257 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: NON_VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 10:08:00.678462 9116 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6b52febb05ad477dac2b5d72e786208c (127.2.30.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: NON_VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: true } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } } }
I20260808 10:08:00.679767 9145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c974e52359c14e1aaf0c36f072c48dcc. Latest consensus state: current_term: 1 leader_uuid: "c974e52359c14e1aaf0c36f072c48dcc" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: NON_VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: true } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } } }
I20260808 10:08:00.679865 9145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: This master's current role is: LEARNER
I20260808 10:08:00.679919 9145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c974e52359c14e1aaf0c36f072c48dcc" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: NON_VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: true } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } } }
I20260808 10:08:00.679970 9145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: This master's current role is: LEARNER
I20260808 10:08:00.680306 9068 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc: attempting to promote NON_VOTER 9f3602e8df67413eaf140373844707a5 to VOTER
I20260808 10:08:00.680513 9068 consensus_queue.cc:237] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } }
I20260808 10:08:00.681062 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.680941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:39837 (local address 127.0.0.1:56938)
0808 10:08:00.680982 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.681034 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 10:08:00.681430 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.681277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:39837 (local address 127.0.0.1:56942)
0808 10:08:00.681384 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.681405 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:39837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":75,"threads_started":1}
W20260808 10:08:00.681483 8472 consensus_peers.cc:487] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc -> Peer 6b52febb05ad477dac2b5d72e786208c (127.2.30.188:39837): Couldn't send request to peer 6b52febb05ad477dac2b5d72e786208c. Status: Network error: Client connection negotiation failed: client connection to 127.2.30.188:39837: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 10:08:00.681794 9116 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 LEARNER]: Refusing update from remote peer c974e52359c14e1aaf0c36f072c48dcc: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 10:08:00.682050 9260 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 10:08:00.682687 9257 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9f3602e8df67413eaf140373844707a5 (127.2.30.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } } }
I20260808 10:08:00.684031 9067 sys_catalog.cc:455] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c974e52359c14e1aaf0c36f072c48dcc" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } } }
I20260808 10:08:00.684116 9067 sys_catalog.cc:458] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:00.684379 9067 sys_catalog.cc:455] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c974e52359c14e1aaf0c36f072c48dcc" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } } }
I20260808 10:08:00.684443 9067 sys_catalog.cc:458] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:00.684810 9116 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9f3602e8df67413eaf140373844707a5 (127.2.30.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } } }
I20260808 10:08:00.685760 9145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c974e52359c14e1aaf0c36f072c48dcc" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } } }
I20260808 10:08:00.685842 9145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:08:00.688135 9160 master.cc:552] Master@127.2.30.188:39837 shutdown complete.
I20260808 10:08:00.688179 9160 master_runner.cc:305] Clearing existing system tablet
I20260808 10:08:00.688324 9160 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:00.688364 9160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:00.688993 9160 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:08:00.689565 9263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:00.689673 9160 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:00.689719 9160 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:00.689740 9160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/wal
uuid: "6b52febb05ad477dac2b5d72e786208c"
format_stamp: "Formatted at 2026-08-08 10:08:00 on dist-test-slave-w206"
server_key: "42be1ea18d8a756cc21401d09b4b8b6b"
server_key_iv: "07250eec200e3ac862d89f201c1975d7"
server_key_version: "encryptionkey@0"
I20260808 10:08:00.690461 9160 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:00.692147 9160 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 10:08:00.692222 9160 log.cc:1196] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 10:08:00.692414 9160 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: Deleting consensus metadata
I20260808 10:08:00.692552 9160 master_runner.cc:315] Copying system tablet from 127.2.30.190:41065
I20260808 10:08:00.693187 9160 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: tablet copy: Beginning tablet copy session from remote peer at address 127.2.30.190:41065
I20260808 10:08:00.705821 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.695308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59696 (local address 127.2.30.190:41065)
0808 10:08:00.695359 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.695362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.698314 (+ 2952us) server_negotiation.cc:407] Connection header received
0808 10:08:00.698385 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.698387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.698404 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.698472 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.698917 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.699778 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.700918 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.701227 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.701304 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.702518 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.702520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.702520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.702528 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.704399 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.704845 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.704847 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.704847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.704917 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.705149 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.705150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.705150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.705304 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.705381 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.705634 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.705723 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:08:00.705830 9269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.695227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:59696)
0808 10:08:00.698191 (+ 2964us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.698206 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.698304 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.698493 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.698494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.698739 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.698810 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.698812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.699877 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.699878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.700819 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.700822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.701318 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.701319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.701391 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.701974 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.701979 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.702421 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.704500 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.704501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.704501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.704765 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.704997 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.704998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.704998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.705072 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.705380 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.705381 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.705437 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.705570 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.705692 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2933,"thread_start_us":78,"threads_started":1}
I20260808 10:08:00.706625 8509 tablet_copy_service.cc:140] P c974e52359c14e1aaf0c36f072c48dcc: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6b52febb05ad477dac2b5d72e786208c ({username='oryx', principal='oryx/127.2.30.188@KRBTEST.COM'} at 127.0.0.1:59696)
I20260808 10:08:00.706692 8509 tablet_copy_service.cc:161] P c974e52359c14e1aaf0c36f072c48dcc: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6b52febb05ad477dac2b5d72e786208c at {username='oryx', principal='oryx/127.2.30.188@KRBTEST.COM'} at 127.0.0.1:59696: session id = 6b52febb05ad477dac2b5d72e786208c-00000000000000000000000000000000
I20260808 10:08:00.707420 8509 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 10:08:00.707954 9160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 10:08:00.709676 9160 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: tablet copy: Starting download of 0 data blocks...
I20260808 10:08:00.709944 9160 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: tablet copy: Starting download of 1 WAL segments...
I20260808 10:08:00.710886 9160 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 10:08:00.712028 8509 tablet_copy_service.cc:342] P c974e52359c14e1aaf0c36f072c48dcc: Request end of tablet copy session 6b52febb05ad477dac2b5d72e786208c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.30.188@KRBTEST.COM'} at 127.0.0.1:59696
I20260808 10:08:00.712092 8509 tablet_copy_service.cc:433] P c974e52359c14e1aaf0c36f072c48dcc: ending tablet copy session 6b52febb05ad477dac2b5d72e786208c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6b52febb05ad477dac2b5d72e786208c
I20260808 10:08:00.715206 9160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:00.716548 9273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:00.716604 9270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:00.719152 9271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:00.719188 9160 server_base.cc:1034] running on GCE node
I20260808 10:08:00.719419 9160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:00.719581 9160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:00.720729 9160 hybrid_clock.cc:648] HybridClock initialized: now 1786183680720732 us; error 69 us; skew 500 ppm
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:00.722426 9160 init.cc:377] Logged in from keytab as oryx/127.2.30.188@KRBTEST.COM (short username oryx)
I20260808 10:08:00.722748 9160 webserver.cc:460] Webserver started at http://127.2.30.188:37061/ using document root <none> and password file <none>
I20260808 10:08:00.722882 9160 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:00.722922 9160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:00.723776 9160 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:08:00.724402 9280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:00.724586 9160 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:08:00.724642 9160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/wal
uuid: "6b52febb05ad477dac2b5d72e786208c"
format_stamp: "Formatted at 2026-08-08 10:08:00 on dist-test-slave-w206"
server_key: "42be1ea18d8a756cc21401d09b4b8b6b"
server_key_iv: "07250eec200e3ac862d89f201c1975d7"
server_key_version: "encryptionkey@0"
I20260808 10:08:00.724709 9160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:00.761878 9160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:00.763935 9160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:00.767383 9160 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.188:39837
I20260808 10:08:00.767426 9337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.188:39837 every 8 connection(s)
I20260808 10:08:00.767736 9160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/master-2/data/info.pb
I20260808 10:08:00.768689 9338 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:08:00.769227 9338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: Bootstrap starting.
W20260808 10:08:00.769464 9256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:00.769678 9256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:00.769722 9256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:00.771211 9256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:00.771274 9256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:00.771333 9256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:08:00.772162 9338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:08:00.772387 9338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: Bootstrap complete.
I20260808 10:08:00.772562 9338 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } }
I20260808 10:08:00.772642 9338 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6b52febb05ad477dac2b5d72e786208c, State: Initialized, Role: LEARNER
I20260808 10:08:00.772675 9338 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } }
I20260808 10:08:00.772843 9338 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:08:00.772909 9341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } } }
I20260808 10:08:00.773070 9341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [sys.catalog]: This master's current role is: LEARNER
I20260808 10:08:00.773320 9256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.129.keytab
--principal=oryx/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:44221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=35427
--tserver_master_addrs=127.2.30.190:41065,127.2.30.189:46359,127.2.30.188:39837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:00.773581 9256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:00.773873 9256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:08:00.774690 9353 catalog_manager.cc:1259] Loaded cluster ID: 8987d372a5fd4b5aa3c86650e67245ab
I20260808 10:08:00.774963 9353 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: loading cluster ID for follower catalog manager: success
I20260808 10:08:00.776117 9353 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: acquiring CA information for follower catalog manager: success
I20260808 10:08:00.776412 9353 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 10:08:00.780339 9355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:00.780365 9356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:00.780529 9256 server_base.cc:1034] running on GCE node
W20260808 10:08:00.780588 9358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:00.780866 9256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:00.781181 9256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:00.782377 9256 hybrid_clock.cc:648] HybridClock initialized: now 1786183680782355 us; error 37 us; skew 500 ppm
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:00.785312 9256 init.cc:377] Logged in from keytab as oryx/127.2.30.129@KRBTEST.COM (short username oryx)
I20260808 10:08:00.785808 9256 webserver.cc:460] Webserver started at http://127.2.30.129:35427/ using document root <none> and password file <none>
I20260808 10:08:00.786067 9256 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:00.786134 9256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:00.787844 9256 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:08:00.788580 9365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:00.788786 9256 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:00.788882 9256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "74d1601b5a52457a951fafd40b699521"
format_stamp: "Formatted at 2026-08-08 10:07:59 on dist-test-slave-w206"
server_key: "3b81ae3287a395ba70001f260317b016"
server_key_iv: "a0f015c668fdc8783c82d18ef10b65d6"
server_key_version: "encryptionkey@0"
I20260808 10:08:00.789309 9256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:00.838078 9256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:00.840636 9256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:00.840759 9256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:00.841033 9256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:00.841444 9256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:00.841499 9256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:00.841535 9256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:00.841563 9256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:00.847664 9256 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:44221
I20260808 10:08:00.847723 9482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:44221 every 8 connection(s)
I20260808 10:08:00.848080 9256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:08:00.850378 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 9256
I20260808 10:08:00.850569 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 8691
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.189@KRBTEST.COM
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM
I20260808 10:08:00.860236 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:36601
--local_ip_for_outbound_sockets=127.2.30.130
--tserver_master_addrs=127.2.30.190:41065
--webserver_port=35333
--webserver_interface=127.2.30.130
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.130.keytab
--principal=oryx/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.30.190:41065,127.2.30.189:46359
--tserver_master_addrs=127.2.30.190:41065,127.2.30.189:46359,127.2.30.188:39837 with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:08:00.861886 9157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.849133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:39211 (local address 127.2.30.189:46359)
0808 10:08:00.849181 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.849183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.849421 (+ 238us) server_negotiation.cc:407] Connection header received
0808 10:08:00.850296 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.850298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.850313 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.850351 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.852433 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.853108 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.854536 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.854873 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.854922 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.858064 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.858066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.858066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.858075 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.860138 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.860720 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.860721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.860722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.860796 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.861143 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.861144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.861144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.861287 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.861375 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.861687 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.861782 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:08:00.861925 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.849088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:48247 (local address 127.2.30.190:41065)
0808 10:08:00.849138 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.849140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.849151 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:08:00.850602 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.850603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.850619 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.850687 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.852455 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.853434 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.856167 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.856474 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.856518 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.858826 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.858828 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.858828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.858842 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.860639 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.861039 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.861040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.861040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.861097 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.861309 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.861310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.861310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.861452 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.861496 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.861700 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.861790 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 10:08:00.862113 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.848824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:46359 (local address 127.2.30.129:39211)
0808 10:08:00.849334 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.849343 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.850172 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.850529 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.850533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.852032 (+ 1499us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.852321 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.852327 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.853243 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.853244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.854430 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.854434 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.855050 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.855051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.855211 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.855821 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.855833 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.857935 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.860249 (+ 2314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.860258 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.860261 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.860610 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.860877 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.860878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.860879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.861057 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.861366 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.861373 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.861534 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.861657 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.861841 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":6,"spinlock_wait_cycles":8064,"thread_start_us":114,"threads_started":1}
I20260808 10:08:00.862274 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.848823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.129:48247)
0808 10:08:00.849022 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.849066 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.850110 (+ 1044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.851518 (+ 1408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.851519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.852140 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.852322 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.852331 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.853543 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.853544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.854663 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.854666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.856598 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.856599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.856664 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.857135 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.857140 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.858722 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.860732 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.860733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.860734 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.860956 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.861167 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.861167 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.861168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.861228 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.861544 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.861545 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.861587 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.862055 (+ 468us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.862171 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.188@KRBTEST.COM
I20260808 10:08:00.862660 9485 heartbeater.cc:347] Connected to a master server at 127.2.30.189:46359
I20260808 10:08:00.862792 9485 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:00.862661 9484 heartbeater.cc:347] Connected to a master server at 127.2.30.190:41065
I20260808 10:08:00.863041 9484 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:00.863199 9485 heartbeater.cc:511] Master 127.2.30.189:46359 requested a full tablet report, sending...
I20260808 10:08:00.863392 9484 heartbeater.cc:511] Master 127.2.30.190:41065 requested a full tablet report, sending...
I20260808 10:08:00.863744 8488 ts_manager.cc:194] Re-registered known tserver with Master: 74d1601b5a52457a951fafd40b699521 (127.2.30.129:44221)
I20260808 10:08:00.863986 9100 ts_manager.cc:194] Registered new tserver with Master: 74d1601b5a52457a951fafd40b699521 (127.2.30.129:44221)
I20260808 10:08:00.864282 8488 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:48247
I20260808 10:08:00.866653 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.848824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:39837 (local address 127.2.30.129:39623)
0808 10:08:00.849275 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.849286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.850162 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.850529 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.850533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.854748 (+ 4215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.854816 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.854818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.856117 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.856118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.857210 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.857213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.857771 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.857773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.857855 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.858276 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.858281 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:00.863030 (+ 4749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:00.865442 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.865443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.865444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.865654 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.865915 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:00.865916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:00.865916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:00.865978 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:00.866348 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:00.866349 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:00.866403 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 10:08:00.866472 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 10:08:00.866576 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":107,"threads_started":1}
I20260808 10:08:00.867230 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.849133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:39623 (local address 127.2.30.188:39837)
0808 10:08:00.849402 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.849404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.849412 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:08:00.850317 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.850318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.850333 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.850413 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.854907 (+ 4494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.856000 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.857312 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.857647 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.857711 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.863147 (+ 5436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:00.863149 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:00.863149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:00.863159 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:00.865346 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.865763 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.865765 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.865765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.865843 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:00.866043 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:00.866044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:00.866044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:00.866198 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:00.866270 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:00.866544 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 10:08:00.866649 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":66,"threads_started":1}
I20260808 10:08:00.867614 9483 heartbeater.cc:347] Connected to a master server at 127.2.30.188:39837
I20260808 10:08:00.867673 9483 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:00.867758 9483 heartbeater.cc:511] Master 127.2.30.188:39837 requested a full tablet report, sending...
I20260808 10:08:00.868192 9298 ts_manager.cc:194] Registered new tserver with Master: 74d1601b5a52457a951fafd40b699521 (127.2.30.129:44221)
W20260808 10:08:00.949721 9491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:00.949949 9491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:00.949975 9491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:00.951172 9491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:00.951215 9491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:00.951242 9491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:08:00.952840 9491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.130.keytab
--principal=oryx/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:36601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=35333
--tserver_master_addrs=127.2.30.190:41065,127.2.30.189:46359,127.2.30.188:39837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:00.953039 9491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:00.953227 9491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:00.958634 9497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:00.958634 9499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:00.958664 9496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:00.958665 9491 server_base.cc:1034] running on GCE node
I20260808 10:08:00.959120 9491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:00.959394 9491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:00.960597 9491 hybrid_clock.cc:648] HybridClock initialized: now 1786183680960576 us; error 51 us; skew 500 ppm
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:00.963344 9491 init.cc:377] Logged in from keytab as oryx/127.2.30.130@KRBTEST.COM (short username oryx)
I20260808 10:08:00.963881 9491 webserver.cc:460] Webserver started at http://127.2.30.130:35333/ using document root <none> and password file <none>
I20260808 10:08:00.964154 9491 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:00.964236 9491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:00.965715 9491 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:00.966526 9507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:00.966744 9491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:00.966850 9491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "a116392f87ac403ab0d65007de2db5cf"
format_stamp: "Formatted at 2026-08-08 10:07:59 on dist-test-slave-w206"
server_key: "6209dec25d496b77366a75033e3f7539"
server_key_iv: "e82775733cdd6495fe92dcc1458a7b1b"
server_key_version: "encryptionkey@0"
I20260808 10:08:00.967286 9491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:00.981600 9491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:00.983831 9491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:00.984007 9491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:00.984269 9491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:00.984602 9491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:00.984656 9491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:00.984730 9491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:00.984762 9491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:00.991192 9491 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:36601
I20260808 10:08:00.991243 9625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:36601 every 8 connection(s)
I20260808 10:08:00.991605 9491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:08:00.999199 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 9491
I20260808 10:08:00.999372 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 8828
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.189@KRBTEST.COM
Aug 08 10:08:00 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183680, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.188@KRBTEST.COM
I20260808 10:08:01.005275 9157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.992404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:48949 (local address 127.2.30.189:46359)
0808 10:08:00.992481 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.992484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.992859 (+ 375us) server_negotiation.cc:407] Connection header received
0808 10:08:00.994091 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.994093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.994107 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.994151 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.995184 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.995746 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.996872 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.997138 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.997228 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.000934 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:01.000935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:01.000936 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:01.000945 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:01.003088 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.003760 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.003761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.003762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.003835 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.004217 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.004219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.004220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.004408 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:01.004463 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.004984 (+ 521us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.005100 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 10:08:01.005434 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.992487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:40509 (local address 127.2.30.188:39837)
0808 10:08:00.992526 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.992528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.992781 (+ 253us) server_negotiation.cc:407] Connection header received
0808 10:08:00.994076 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.994078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.994098 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.994170 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.995206 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.995965 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.997111 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.997410 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.997498 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.001815 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:01.001817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:01.001817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:01.001828 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:01.003841 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.004356 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.004358 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.004358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.004426 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.004667 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.004668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.004668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.004826 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:01.004876 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.005170 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.005298 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:08:01.006017 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.992414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:46359 (local address 127.2.30.130:48949)
0808 10:08:00.992773 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.992786 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.993930 (+ 1144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.994256 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.994257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.994795 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.995075 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.995081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.995873 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.995874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.996768 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.996771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.997234 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.997235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.997374 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.998003 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.998033 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:01.000806 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:01.003211 (+ 2405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:01.003223 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:01.003227 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:01.003653 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:01.003921 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:01.003922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:01.003922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:01.004104 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:01.004519 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:01.004528 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:01.004777 (+ 249us) client_negotiation.cc:769] Sending connection context
0808 10:08:01.005401 (+ 624us) client_negotiation.cc:240] Negotiation successful
0808 10:08:01.005657 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":4,"thread_start_us":111,"threads_started":1}
I20260808 10:08:01.006093 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.993001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:47915 (local address 127.2.30.190:41065)
0808 10:08:00.993076 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:00.993078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:00.993111 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:08:00.994318 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:00.994319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:00.994334 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:00.994399 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:00.995193 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.995974 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.997129 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.997425 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.997492 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.001815 (+ 4323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:01.001817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:01.001817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:01.001827 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:01.004679 (+ 2852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.005149 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.005150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.005151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.005218 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.005441 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.005442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.005442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.005601 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:01.005660 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.005898 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.005987 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:08:01.006618 9632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.992796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.130:47915)
0808 10:08:00.993061 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.993069 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.993921 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.994414 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.994415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.994795 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.995077 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.995083 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.996069 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.996070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.997021 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.997022 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.997562 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.997563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.997633 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.998114 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.998118 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:01.001693 (+ 3575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:01.004817 (+ 3124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:01.004818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:01.004818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:01.005048 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:01.005305 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:01.005306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:01.005306 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:01.005367 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:01.005738 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:01.005739 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:01.005783 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 10:08:01.006367 (+ 584us) client_negotiation.cc:240] Negotiation successful
0808 10:08:01.006473 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":82,"thread_start_us":71,"threads_started":1}
I20260808 10:08:01.006840 9631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:00.992414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:39837 (local address 127.2.30.130:40509)
0808 10:08:00.992609 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:00.992656 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:00.993910 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:00.994247 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:00.994250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:00.994795 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:00.995077 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.995083 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.996069 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.996070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.997018 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:00.997021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:00.997497 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:00.997498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:00.997583 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:00.998321 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:00.998332 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:01.001687 (+ 3355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:01.003996 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:01.003997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:01.003998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:01.004249 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:01.004514 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:01.004516 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:01.004516 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:01.004586 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:01.004928 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:01.004929 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:01.004976 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 10:08:01.005949 (+ 973us) client_negotiation.cc:240] Negotiation successful
0808 10:08:01.006065 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":63,"thread_start_us":111,"threads_started":1}
I20260808 10:08:01.007098 9627 heartbeater.cc:347] Connected to a master server at 127.2.30.190:41065
I20260808 10:08:01.007252 9627 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:01.007537 9627 heartbeater.cc:511] Master 127.2.30.190:41065 requested a full tablet report, sending...
I20260808 10:08:01.008009 8488 ts_manager.cc:194] Re-registered known tserver with Master: a116392f87ac403ab0d65007de2db5cf (127.2.30.130:36601)
I20260808 10:08:01.008424 8488 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:47915
I20260808 10:08:01.011602 9626 heartbeater.cc:347] Connected to a master server at 127.2.30.188:39837
I20260808 10:08:01.011668 9626 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:01.011758 9626 heartbeater.cc:511] Master 127.2.30.188:39837 requested a full tablet report, sending...
I20260808 10:08:01.011924 9628 heartbeater.cc:347] Connected to a master server at 127.2.30.189:46359
I20260808 10:08:01.011965 9628 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:01.012008 9628 heartbeater.cc:511] Master 127.2.30.189:46359 requested a full tablet report, sending...
I20260808 10:08:01.012109 9298 ts_manager.cc:194] Registered new tserver with Master: a116392f87ac403ab0d65007de2db5cf (127.2.30.130:36601)
I20260808 10:08:01.012370 9100 ts_manager.cc:194] Registered new tserver with Master: a116392f87ac403ab0d65007de2db5cf (127.2.30.130:36601)
I20260808 10:08:01.015630 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:33039
--local_ip_for_outbound_sockets=127.2.30.131
--tserver_master_addrs=127.2.30.190:41065
--webserver_port=42459
--webserver_interface=127.2.30.131
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.131.keytab
--principal=oryx/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:32907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.30.190:41065,127.2.30.189:46359
--tserver_master_addrs=127.2.30.190:41065,127.2.30.189:46359,127.2.30.188:39837 with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:01.103991 9633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:01.104211 9633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:01.104238 9633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:01.105579 9633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:01.105623 9633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:01.105651 9633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:08:01.107280 9633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:32907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.131.keytab
--principal=oryx/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:33039
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=42459
--tserver_master_addrs=127.2.30.190:41065,127.2.30.189:46359,127.2.30.188:39837
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:01.107547 9633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:01.107820 9633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:01.113186 9639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:01.113193 9641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:01.113193 9638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:01.113587 9633 server_base.cc:1034] running on GCE node
I20260808 10:08:01.113773 9633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:01.114014 9633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:01.115172 9633 hybrid_clock.cc:648] HybridClock initialized: now 1786183681115158 us; error 29 us; skew 500 ppm
Aug 08 10:08:01 dist-test-slave-w206 krb5kdc[8442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183681, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:01.117889 9633 init.cc:377] Logged in from keytab as oryx/127.2.30.131@KRBTEST.COM (short username oryx)
I20260808 10:08:01.118309 9633 webserver.cc:460] Webserver started at http://127.2.30.131:42459/ using document root <none> and password file <none>
I20260808 10:08:01.118571 9633 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:01.118666 9633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:01.120198 9633 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:01.120877 9648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:01.121057 9633 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:01.121165 9633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "71379382edcc4184aa937dbb8098251e"
format_stamp: "Formatted at 2026-08-08 10:08:00 on dist-test-slave-w206"
server_key: "97c0ae2a0bd1b606e9fdf04f1bb98463"
server_key_iv: "942359ef67430f20a76e9da1cd7d1202"
server_key_version: "encryptionkey@0"
I20260808 10:08:01.121553 9633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:01.152482 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.149417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56948 (local address 127.2.30.188:39837)
0808 10:08:01.149488 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:01.149490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:01.149503 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:08:01.149554 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:01.149555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:01.149576 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:01.149666 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:01.149833 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.150336 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.151362 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.151902 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.152038 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.152135 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.152361 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.152408 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:08:01.152482 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.149291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:39837 (local address 127.0.0.1:56948)
0808 10:08:01.149368 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:01.149383 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:01.149473 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:01.149673 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:01.149674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:01.149676 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:01.149733 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.149735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.150434 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.150435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.151264 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.151267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.152102 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.152103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.152176 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.152255 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 10:08:01.152339 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:08:01.152396 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 10:08:01.152895 9308 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 10:08:01.153077 9341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c974e52359c14e1aaf0c36f072c48dcc. Latest consensus state: current_term: 1 leader_uuid: "c974e52359c14e1aaf0c36f072c48dcc" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } } }
I20260808 10:08:01.153199 9341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [sys.catalog]: This master's current role is: LEARNER
I20260808 10:08:01.153991 9633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:01.156487 9633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:01.156607 9633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:01.156905 9633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:01.157258 9633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:01.157294 9633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:01.157318 9633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:01.157366 9633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:01.163659 9633 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:33039
I20260808 10:08:01.163776 9766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:33039 every 8 connection(s)
I20260808 10:08:01.164165 9633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
Aug 08 10:08:01 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183681, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM
Aug 08 10:08:01 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183681, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.189@KRBTEST.COM
I20260808 10:08:01.172045 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 9633
Aug 08 10:08:01 dist-test-slave-w206 krb5kdc[8442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183681, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.188@KRBTEST.COM
I20260808 10:08:01.176915 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.165426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:44893 (local address 127.2.30.190:41065)
0808 10:08:01.165474 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:01.165476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:01.165555 (+ 79us) server_negotiation.cc:407] Connection header received
0808 10:08:01.166783 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:01.166784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:01.166794 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:01.166860 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:01.167542 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.168051 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.169145 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.169344 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.169422 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.172217 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:01.172219 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:01.172220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:01.172229 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:01.174218 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.175777 (+ 1559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.175778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.175778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.175835 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.176162 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.176163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.176163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.176305 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:01.176362 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.176718 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.176823 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:08:01.176967 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.165092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:53355 (local address 127.2.30.188:39837)
0808 10:08:01.165132 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:01.165135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:01.165377 (+ 242us) server_negotiation.cc:407] Connection header received
0808 10:08:01.166671 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:01.166672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:01.166687 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:01.166729 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:01.167558 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.168318 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.169944 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.170234 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.170301 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.173554 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:01.173556 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:01.173556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:01.173580 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:01.175460 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.175858 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.175859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.175860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.175918 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.176142 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.176142 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.176143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.176281 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:01.176336 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.176767 (+ 431us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.176852 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:08:01.177112 9771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.164985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:46359 (local address 127.2.30.131:60975)
0808 10:08:01.165224 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:01.165249 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:01.166398 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:01.166667 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:01.166671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:01.167199 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:01.167437 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.167443 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.168442 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.168443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.169383 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.169385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.169890 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.169891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.169962 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.170482 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:01.170487 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:01.172316 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:01.175263 (+ 2947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:01.175264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:01.175265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:01.175672 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:01.175908 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:01.175909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:01.175909 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:01.176074 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:01.176501 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:01.176502 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:01.176553 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 10:08:01.176668 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 10:08:01.176860 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":105,"spinlock_wait_cycles":9472,"thread_start_us":88,"threads_started":1}
I20260808 10:08:01.177255 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.165365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.131:44893)
0808 10:08:01.165496 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:01.165505 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:01.166417 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:01.166868 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:01.166870 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:01.167209 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:01.167436 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.167441 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.168161 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.168162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.169037 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.169040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.169428 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.169430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.169572 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.170273 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:01.170284 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:01.172092 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:01.175173 (+ 3081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:01.175183 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:01.175186 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:01.175672 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:01.175909 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:01.175909 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:01.175909 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:01.176074 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:01.176396 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:01.176397 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:01.176553 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 10:08:01.176995 (+ 442us) client_negotiation.cc:240] Negotiation successful
0808 10:08:01.177088 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":2,"thread_start_us":69,"threads_started":1}
I20260808 10:08:01.177384 9772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.164985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:39837 (local address 127.2.30.131:53355)
0808 10:08:01.165202 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:01.165249 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:01.166398 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:01.166768 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:01.166769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:01.167206 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:01.167437 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.167443 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.168432 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.168433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.169389 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.169390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.170355 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.170356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.170428 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.170852 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:01.170857 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:01.173446 (+ 2589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:01.175551 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:01.175551 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:01.175552 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:01.175763 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:01.175986 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:01.175986 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:01.175987 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:01.176077 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:01.176364 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:01.176371 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:01.176554 (+ 183us) client_negotiation.cc:769] Sending connection context
0808 10:08:01.176668 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 10:08:01.177195 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":407,"thread_start_us":117,"threads_started":1}
I20260808 10:08:01.178128 9767 heartbeater.cc:347] Connected to a master server at 127.2.30.188:39837
I20260808 10:08:01.178249 9767 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:01.178447 9157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.164985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:60975 (local address 127.2.30.189:46359)
0808 10:08:01.165041 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:01.165043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:01.165349 (+ 306us) server_negotiation.cc:407] Connection header received
0808 10:08:01.166523 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:01.166525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:01.166541 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:01.166597 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:01.167557 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.168343 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.169501 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.169817 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.169872 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.172439 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:01.172440 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:01.172441 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:01.172448 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:01.175068 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.175761 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.175763 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.175763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.175831 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.176279 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.176280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.176280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.176421 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:01.176478 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.178283 (+ 1805us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.178379 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 10:08:01.178499 9767 heartbeater.cc:511] Master 127.2.30.188:39837 requested a full tablet report, sending...
I20260808 10:08:01.178810 9769 heartbeater.cc:347] Connected to a master server at 127.2.30.189:46359
I20260808 10:08:01.178861 9769 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:01.178907 9768 heartbeater.cc:347] Connected to a master server at 127.2.30.190:41065
I20260808 10:08:01.178931 9769 heartbeater.cc:511] Master 127.2.30.189:46359 requested a full tablet report, sending...
I20260808 10:08:01.178938 9768 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:01.179034 9768 heartbeater.cc:511] Master 127.2.30.190:41065 requested a full tablet report, sending...
I20260808 10:08:01.179085 9298 ts_manager.cc:194] Registered new tserver with Master: 71379382edcc4184aa937dbb8098251e (127.2.30.131:33039)
I20260808 10:08:01.179262 9100 ts_manager.cc:194] Registered new tserver with Master: 71379382edcc4184aa937dbb8098251e (127.2.30.131:33039)
I20260808 10:08:01.179314 8488 ts_manager.cc:194] Re-registered known tserver with Master: 71379382edcc4184aa937dbb8098251e (127.2.30.131:33039)
I20260808 10:08:01.179800 8488 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:44893
I20260808 10:08:01.186560 9779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.173593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56960 (local address 127.2.30.188:39837)
0808 10:08:01.174258 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:01.174260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:01.174269 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:08:01.174291 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:01.174291 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:01.174302 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:01.174379 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:01.175556 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.176351 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.180631 (+ 4280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.180946 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.181019 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.182960 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:01.182961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:01.182962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:01.183015 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:01.184923 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.185432 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.185433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.185433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.185501 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.185738 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.185739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.185740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.185896 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:01.185999 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.186213 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.186308 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":60,"threads_started":1}
I20260808 10:08:01.186735 9782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.174177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59702 (local address 127.2.30.190:41065)
0808 10:08:01.174675 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:01.174677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:01.174685 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:08:01.174705 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:01.174706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:01.174719 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:01.177065 (+ 2346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:01.177107 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.177894 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.181464 (+ 3570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.181729 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.181829 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.183311 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:01.183312 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:01.183312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:01.183320 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:01.185144 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.185621 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.185622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.185623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.185711 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.185923 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.185924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.185924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.186074 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:01.186494 (+ 420us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.186533 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.186612 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":110,"threads_started":1}
I20260808 10:08:01.187045 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.173993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46276 (local address 127.2.30.189:46359)
0808 10:08:01.174454 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:01.174456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:01.174885 (+ 429us) server_negotiation.cc:407] Connection header received
0808 10:08:01.176947 (+ 2062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:01.176949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:01.176965 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:01.177006 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:01.179668 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.180299 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.181540 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.181735 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.181818 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.183472 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:01.183473 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:01.183474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:01.183480 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:01.185262 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.185700 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.185701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.185701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.185770 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.186013 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.186014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.186014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.186154 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:01.186239 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.186419 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.186510 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":86,"threads_started":1}
I20260808 10:08:01.188465 8488 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:08:01.190001 8488 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:08:01.190719 9260 sys_catalog.cc:455] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c974e52359c14e1aaf0c36f072c48dcc" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: NON_VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: true } } }
I20260808 10:08:01.190825 9260 sys_catalog.cc:458] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:01.192325 9260 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc: attempting to promote NON_VOTER 6b52febb05ad477dac2b5d72e786208c to VOTER
I20260808 10:08:01.192459 9260 consensus_queue.cc:237] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } }
I20260808 10:08:01.192833 9116 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 FOLLOWER]: Refusing update from remote peer c974e52359c14e1aaf0c36f072c48dcc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 10:08:01.193153 9307 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 1 LEARNER]: Refusing update from remote peer c974e52359c14e1aaf0c36f072c48dcc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 10:08:01.193141 9786 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 10:08:01.193540 9787 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 10:08:01.193929 9787 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6b52febb05ad477dac2b5d72e786208c (127.2.30.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } } }
I20260808 10:08:01.194197 9116 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6b52febb05ad477dac2b5d72e786208c (127.2.30.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } } }
I20260808 10:08:01.194368 9307 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6b52febb05ad477dac2b5d72e786208c (127.2.30.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } } }
I20260808 10:08:01.195237 9787 sys_catalog.cc:455] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c974e52359c14e1aaf0c36f072c48dcc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } } }
I20260808 10:08:01.195345 9787 sys_catalog.cc:458] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:01.195423 9787 sys_catalog.cc:455] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c974e52359c14e1aaf0c36f072c48dcc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } } }
I20260808 10:08:01.195473 9787 sys_catalog.cc:458] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:01.195489 9786 sys_catalog.cc:455] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c974e52359c14e1aaf0c36f072c48dcc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } } }
I20260808 10:08:01.195580 9786 sys_catalog.cc:458] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:01.195654 9341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c974e52359c14e1aaf0c36f072c48dcc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } } }
I20260808 10:08:01.195796 9341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:08:01.197116 9796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c974e52359c14e1aaf0c36f072c48dcc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } } }
I20260808 10:08:01.197228 9796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:08:01.200525 9798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.195957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:33039 (local address 127.0.0.1:45056)
0808 10:08:01.196705 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:01.196711 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:01.196783 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:01.197284 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:01.197285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:01.197288 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:01.197334 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.197337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.198475 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.198476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.199595 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.199597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.200274 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.200275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.200336 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.200341 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:08:01.200397 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 10:08:01.200442 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":81,"threads_started":1}
I20260808 10:08:01.200541 9801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.196194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45056 (local address 127.2.30.131:33039)
0808 10:08:01.196365 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:01.196369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:01.196778 (+ 409us) server_negotiation.cc:407] Connection header received
0808 10:08:01.197042 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:01.197044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:01.197062 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:01.197239 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:01.197425 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.198397 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.199658 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.200186 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.200248 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.200308 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.200417 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.200456 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":80,"threads_started":1}
I20260808 10:08:01.201545 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.195695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:36601 (local address 127.0.0.1:42212)
0808 10:08:01.195743 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:01.195755 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:01.195795 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:01.197342 (+ 1547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:01.197343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:01.197345 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:01.197389 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.197390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.198234 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.198236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.199427 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.199429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.201307 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.201308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.201373 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.201377 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:08:01.201434 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 10:08:01.201474 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 10:08:01.201567 9797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.195799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42212 (local address 127.2.30.130:36601)
0808 10:08:01.196922 (+ 1123us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:01.196927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:01.196936 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:08:01.196978 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:01.196980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:01.196997 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:01.197346 (+ 349us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:01.197450 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.198138 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.200435 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.201217 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.201304 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.201371 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.201449 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.201490 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1080,"thread_start_us":101,"threads_started":1}
I20260808 10:08:01.201499 9698 tablet_service.cc:1467] Processing CreateTablet for tablet c50b68f2d56b4160b4bf6a46d51dafbf (DEFAULT_TABLE table=test-table [id=451d9a760d214025b1372bdafca7c757]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:01.201884 9799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.195811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:44221 (local address 127.0.0.1:45568)
0808 10:08:01.196584 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:01.196594 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:01.196647 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:01.196822 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:01.196823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:01.196826 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:01.196882 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.196885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.199516 (+ 2631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.199516 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.200329 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.200330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.201615 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.201616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.201682 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.201685 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:08:01.201750 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 10:08:01.201790 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":88,"threads_started":1}
I20260808 10:08:01.201908 9698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c50b68f2d56b4160b4bf6a46d51dafbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:01.201922 9800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.196040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45568 (local address 127.2.30.129:44221)
0808 10:08:01.196206 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:01.196211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:01.196663 (+ 452us) server_negotiation.cc:407] Connection header received
0808 10:08:01.196705 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:01.196707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:01.196722 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:01.196827 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:01.196940 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.198127 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.200419 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.201125 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.201196 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.201572 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.201790 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.201836 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260808 10:08:01.202461 9415 tablet_service.cc:1467] Processing CreateTablet for tablet c50b68f2d56b4160b4bf6a46d51dafbf (DEFAULT_TABLE table=test-table [id=451d9a760d214025b1372bdafca7c757]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:01.202770 9415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c50b68f2d56b4160b4bf6a46d51dafbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:01.203689 9556 tablet_service.cc:1467] Processing CreateTablet for tablet c50b68f2d56b4160b4bf6a46d51dafbf (DEFAULT_TABLE table=test-table [id=451d9a760d214025b1372bdafca7c757]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:01.204102 9556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c50b68f2d56b4160b4bf6a46d51dafbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:01.205456 9803 tablet_bootstrap.cc:492] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e: Bootstrap starting.
I20260808 10:08:01.206074 9803 tablet_bootstrap.cc:654] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:01.206429 9804 tablet_bootstrap.cc:492] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521: Bootstrap starting.
I20260808 10:08:01.206638 9803 log.cc:824] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:01.207106 9804 tablet_bootstrap.cc:654] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:01.207449 9804 log.cc:824] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:01.207509 9803 tablet_bootstrap.cc:492] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e: No bootstrap required, opened a new log
I20260808 10:08:01.207602 9803 ts_tablet_manager.cc:1397] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:08:01.208001 9806 tablet_bootstrap.cc:492] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf: Bootstrap starting.
I20260808 10:08:01.208231 9804 tablet_bootstrap.cc:492] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521: No bootstrap required, opened a new log
I20260808 10:08:01.208304 9804 ts_tablet_manager.cc:1397] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 10:08:01.208617 9806 tablet_bootstrap.cc:654] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:01.208946 9806 log.cc:824] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:01.209638 9806 tablet_bootstrap.cc:492] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf: No bootstrap required, opened a new log
I20260808 10:08:01.209729 9806 ts_tablet_manager.cc:1397] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:08:01.210112 9803 raft_consensus.cc:348] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:01.210309 9803 raft_consensus.cc:374] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:01.210350 9803 raft_consensus.cc:729] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71379382edcc4184aa937dbb8098251e, State: Initialized, Role: FOLLOWER
I20260808 10:08:01.210377 9804 raft_consensus.cc:348] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:01.210520 9804 raft_consensus.cc:374] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:01.210501 9803 consensus_queue.cc:260] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:01.210561 9804 raft_consensus.cc:729] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74d1601b5a52457a951fafd40b699521, State: Initialized, Role: FOLLOWER
I20260808 10:08:01.210685 9804 consensus_queue.cc:260] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:01.210860 9803 ts_tablet_manager.cc:1428] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:08:01.210958 9804 ts_tablet_manager.cc:1428] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 10:08:01.211179 9768 heartbeater.cc:503] Master 127.2.30.190:41065 was elected leader, sending a full tablet report...
I20260808 10:08:01.212252 9806 raft_consensus.cc:348] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:01.212420 9806 raft_consensus.cc:374] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:01.212466 9806 raft_consensus.cc:729] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a116392f87ac403ab0d65007de2db5cf, State: Initialized, Role: FOLLOWER
I20260808 10:08:01.212576 9806 consensus_queue.cc:260] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:01.212829 9806 ts_tablet_manager.cc:1428] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:08:01.212898 9627 heartbeater.cc:503] Master 127.2.30.190:41065 was elected leader, sending a full tablet report...
I20260808 10:08:01.213111 9484 heartbeater.cc:503] Master 127.2.30.190:41065 was elected leader, sending a full tablet report...
W20260808 10:08:01.242451 9629 tablet.cc:2369] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:08:01.349699 9486 tablet.cc:2369] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:08:01.383414 9811 raft_consensus.cc:482] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:08:01.383558 9811 raft_consensus.cc:504] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:01.384042 9811 leader_election.cc:288] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74d1601b5a52457a951fafd40b699521 (127.2.30.129:44221), 71379382edcc4184aa937dbb8098251e (127.2.30.131:33039)
I20260808 10:08:01.388334 9800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.384378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:57807 (local address 127.2.30.129:44221)
0808 10:08:01.384430 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:01.384432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:01.384448 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:08:01.384481 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:01.384482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:01.384495 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:01.384561 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:01.384825 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.385583 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.387335 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.387905 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.387985 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.387995 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.388215 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.388258 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:08:01.388334 9801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.384400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:54911 (local address 127.2.30.131:33039)
0808 10:08:01.384443 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:01.384445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:01.384474 (+ 29us) server_negotiation.cc:407] Connection header received
0808 10:08:01.384498 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:01.384499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:01.384509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:01.384561 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:01.384823 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.385583 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.387336 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.387903 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.387985 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.387995 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.388214 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.388263 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:08:01.388350 9631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.384238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:44221 (local address 127.2.30.130:57807)
0808 10:08:01.384333 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:01.384346 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:01.384400 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:01.384588 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:01.384589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:01.384590 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:01.384714 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.384717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.385706 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.385707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.387231 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.387233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.388024 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.388025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.388118 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.388123 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:08:01.388196 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:08:01.388244 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":63}
I20260808 10:08:01.388361 9632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.384255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:33039 (local address 127.2.30.130:54911)
0808 10:08:01.384365 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:01.384375 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:01.384417 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:01.384588 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:01.384589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:01.384590 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:01.384722 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.384723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.385690 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.385691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.387225 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:01.387230 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.388024 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.388025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:01.388118 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.388123 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:08:01.388240 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 10:08:01.388275 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":111}
I20260808 10:08:01.388798 9436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c50b68f2d56b4160b4bf6a46d51dafbf" candidate_uuid: "a116392f87ac403ab0d65007de2db5cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74d1601b5a52457a951fafd40b699521" is_pre_election: true
I20260808 10:08:01.389008 9436 raft_consensus.cc:2454] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a116392f87ac403ab0d65007de2db5cf in term 0.
I20260808 10:08:01.389067 9719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c50b68f2d56b4160b4bf6a46d51dafbf" candidate_uuid: "a116392f87ac403ab0d65007de2db5cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71379382edcc4184aa937dbb8098251e" is_pre_election: true
I20260808 10:08:01.389245 9719 raft_consensus.cc:2454] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a116392f87ac403ab0d65007de2db5cf in term 0.
I20260808 10:08:01.389279 9508 leader_election.cc:302] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74d1601b5a52457a951fafd40b699521, a116392f87ac403ab0d65007de2db5cf; no voters:
I20260808 10:08:01.389436 9811 raft_consensus.cc:2781] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:08:01.389494 9811 raft_consensus.cc:482] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:08:01.389518 9811 raft_consensus.cc:3037] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:01.390300 9811 raft_consensus.cc:504] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:01.390461 9811 leader_election.cc:288] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [CANDIDATE]: Term 1 election: Requested vote from peers 74d1601b5a52457a951fafd40b699521 (127.2.30.129:44221), 71379382edcc4184aa937dbb8098251e (127.2.30.131:33039)
I20260808 10:08:01.390596 9436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c50b68f2d56b4160b4bf6a46d51dafbf" candidate_uuid: "a116392f87ac403ab0d65007de2db5cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74d1601b5a52457a951fafd40b699521"
I20260808 10:08:01.390642 9719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c50b68f2d56b4160b4bf6a46d51dafbf" candidate_uuid: "a116392f87ac403ab0d65007de2db5cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71379382edcc4184aa937dbb8098251e"
I20260808 10:08:01.390681 9436 raft_consensus.cc:3037] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:01.390753 9719 raft_consensus.cc:3037] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:01.391544 9436 raft_consensus.cc:2454] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a116392f87ac403ab0d65007de2db5cf in term 1.
I20260808 10:08:01.391592 9719 raft_consensus.cc:2454] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a116392f87ac403ab0d65007de2db5cf in term 1.
I20260808 10:08:01.391772 9509 leader_election.cc:302] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71379382edcc4184aa937dbb8098251e, a116392f87ac403ab0d65007de2db5cf; no voters:
I20260808 10:08:01.391891 9811 raft_consensus.cc:2781] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:08:01.392046 9811 raft_consensus.cc:686] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [term 1 LEADER]: Becoming Leader. State: Replica: a116392f87ac403ab0d65007de2db5cf, State: Running, Role: LEADER
I20260808 10:08:01.392161 9811 consensus_queue.cc:237] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:01.393110 8488 catalog_manager.cc:5495] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf reported cstate change: term changed from 0 to 1, leader changed from <none> to a116392f87ac403ab0d65007de2db5cf (127.2.30.130). New cstate: current_term: 1 leader_uuid: "a116392f87ac403ab0d65007de2db5cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } health_report { overall_health: HEALTHY } } }
W20260808 10:08:01.415119 9770 tablet.cc:2369] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:08:01.427891 9797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.424615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42222 (local address 127.2.30.130:36601)
0808 10:08:01.424673 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:01.424675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:01.424751 (+ 76us) server_negotiation.cc:407] Connection header received
0808 10:08:01.424838 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:01.424838 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:01.424862 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:01.424935 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:08:01.425284 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.425921 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.426822 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.427128 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.427257 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.427563 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:08:01.427683 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:08:01.427761 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.427800 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.427822 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:08:01.430011 9719 raft_consensus.cc:1261] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e [term 1 FOLLOWER]: Refusing update from remote peer a116392f87ac403ab0d65007de2db5cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:08:01.430102 9436 raft_consensus.cc:1261] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521 [term 1 FOLLOWER]: Refusing update from remote peer a116392f87ac403ab0d65007de2db5cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:08:01.430447 9811 consensus_queue.cc:1035] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:08:01.430637 9812 consensus_queue.cc:1035] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:08:01.433017 9815 mvcc.cc:205] Tried to move back new op lower bound from 7316208359134081024 to 7316208358983016448. Current Snapshot: MvccSnapshot[applied={T|T < 7316208359134081024}]
I20260808 10:08:01.435045 9816 mvcc.cc:205] Tried to move back new op lower bound from 7316208359134081024 to 7316208358983016448. Current Snapshot: MvccSnapshot[applied={T|T < 7316208359134081024 or (T in {7316208359134081024})}]
I20260808 10:08:01.435725 9819 mvcc.cc:205] Tried to move back new op lower bound from 7316208359134081024 to 7316208358983016448. Current Snapshot: MvccSnapshot[applied={T|T < 7316208359134081024}]
I20260808 10:08:01.437383 8488 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:08:01.437497 8488 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:08:01.439060 8488 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c974e52359c14e1aaf0c36f072c48dcc: Sending DeleteTablet for 3 replicas of tablet c50b68f2d56b4160b4bf6a46d51dafbf
I20260808 10:08:01.439445 9698 tablet_service.cc:1514] Processing DeleteTablet for tablet c50b68f2d56b4160b4bf6a46d51dafbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:01 UTC) from {username='oryx', principal='oryx/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:45056
I20260808 10:08:01.439477 9415 tablet_service.cc:1514] Processing DeleteTablet for tablet c50b68f2d56b4160b4bf6a46d51dafbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:01 UTC) from {username='oryx', principal='oryx/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:45568
I20260808 10:08:01.439623 9556 tablet_service.cc:1514] Processing DeleteTablet for tablet c50b68f2d56b4160b4bf6a46d51dafbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:01 UTC) from {username='oryx', principal='oryx/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:42212
I20260808 10:08:01.439703 9825 tablet_replica.cc:330] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e: stopping tablet replica
I20260808 10:08:01.439807 9825 raft_consensus.cc:2229] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:08:01.439934 9825 raft_consensus.cc:2258] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:01.439934 9827 tablet_replica.cc:330] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf: stopping tablet replica
I20260808 10:08:01.440045 9827 raft_consensus.cc:2229] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:08:01.440095 9826 tablet_replica.cc:330] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521: stopping tablet replica
I20260808 10:08:01.440150 9827 raft_consensus.cc:2258] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:01.440204 9826 raft_consensus.cc:2229] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:08:01.440312 9826 raft_consensus.cc:2258] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:01.441313 9825 ts_tablet_manager.cc:1905] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:01.441267 9827 ts_tablet_manager.cc:1905] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:01.442103 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 8455
I20260808 10:08:01.443377 9826 ts_tablet_manager.cc:1905] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:01.444635 9827 ts_tablet_manager.cc:1918] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:08:01.444721 9827 log.cc:1196] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/c50b68f2d56b4160b4bf6a46d51dafbf
I20260808 10:08:01.445030 9827 ts_tablet_manager.cc:1939] T c50b68f2d56b4160b4bf6a46d51dafbf P a116392f87ac403ab0d65007de2db5cf: Deleting consensus metadata
I20260808 10:08:01.445297 9826 ts_tablet_manager.cc:1918] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:08:01.445369 9826 log.cc:1196] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-0/wal/wals/c50b68f2d56b4160b4bf6a46d51dafbf
I20260808 10:08:01.445652 9826 ts_tablet_manager.cc:1939] T c50b68f2d56b4160b4bf6a46d51dafbf P 74d1601b5a52457a951fafd40b699521: Deleting consensus metadata
I20260808 10:08:01.451805 9825 ts_tablet_manager.cc:1918] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:08:01.451977 9825 log.cc:1196] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/c50b68f2d56b4160b4bf6a46d51dafbf
I20260808 10:08:01.452378 9825 ts_tablet_manager.cc:1939] T c50b68f2d56b4160b4bf6a46d51dafbf P 71379382edcc4184aa937dbb8098251e: Deleting consensus metadata
I20260808 10:08:01.452548 9779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.440754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56962 (local address 127.2.30.188:39837)
0808 10:08:01.441043 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:01.441046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:01.441056 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:08:01.442219 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:01.442221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:01.442240 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:01.442275 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:01.443342 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.444348 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.445724 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.446142 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.446213 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.448915 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:01.448917 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:01.448917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:01.448927 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:01.451029 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.451517 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.451519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.451519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.451590 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.451830 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.451831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.451832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.451991 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:01.452077 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.452333 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.452442 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250}
W20260808 10:08:01.454367 9369 connection.cc:535] server connection from 127.0.0.1:45568 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 10:08:01.454356 9508 connection.cc:535] server connection from 127.0.0.1:42212 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 10:08:01.454399 9649 connection.cc:535] server connection from 127.0.0.1:45056 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 10:08:01.454632 9091 connection.cc:535] server connection from 127.0.0.1:46264 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:08:01.465008 9779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.456643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56976 (local address 127.2.30.188:39837)
0808 10:08:01.456697 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:01.456700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:01.456849 (+ 149us) server_negotiation.cc:407] Connection header received
0808 10:08:01.456957 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:01.456958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:01.456968 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:01.457004 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:01.457978 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.458854 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.459715 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.459981 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.460061 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.461756 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:01.461757 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:01.461757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:01.461764 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:01.463590 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.464011 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.464012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.464013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.464070 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.464319 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.464320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.464320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.464477 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:01.464571 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.464797 (+ 226us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.464896 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 10:08:01.465896 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:01.456720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46286 (local address 127.2.30.189:46359)
0808 10:08:01.456768 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:01.456770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:01.456835 (+ 65us) server_negotiation.cc:407] Connection header received
0808 10:08:01.456927 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:01.456928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:01.456943 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:01.457002 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:01.459011 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.459854 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.460818 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:01.461041 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:01.461136 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:01.462548 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:01.462551 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:01.462551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:01.462559 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:01.464392 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.464829 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.464830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.464831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.464889 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:01.465207 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:01.465209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:01.465209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:01.465344 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:01.465419 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:01.465700 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 10:08:01.465795 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:08:02.434813 9836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:02.434487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.131:45683)
0808 10:08:02.434639 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:02.434736 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
W20260808 10:08:02.435117 9768 heartbeater.cc:644] Failed to heartbeat to 127.2.30.190:41065 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
I20260808 10:08:02.436861 9837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:02.436532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.130:52149)
0808 10:08:02.436696 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:02.436802 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":80,"threads_started":1}
I20260808 10:08:02.437007 9838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:02.436692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.129:39443)
0808 10:08:02.436888 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:02.436961 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":71,"threads_started":1}
W20260808 10:08:02.437098 9627 heartbeater.cc:644] Failed to heartbeat to 127.2.30.190:41065 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
W20260808 10:08:02.437186 9484 heartbeater.cc:644] Failed to heartbeat to 127.2.30.190:41065 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
I20260808 10:08:02.939440 9839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c974e52359c14e1aaf0c36f072c48dcc)
I20260808 10:08:02.939586 9839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } }
I20260808 10:08:02.939941 9839 leader_election.cc:288] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c974e52359c14e1aaf0c36f072c48dcc (127.2.30.190:41065), 6b52febb05ad477dac2b5d72e786208c (127.2.30.188:39837)
I20260808 10:08:02.940076 9840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c974e52359c14e1aaf0c36f072c48dcc)
I20260808 10:08:02.940196 9840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } }
I20260808 10:08:02.940347 9841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:02.940078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:60172)
0808 10:08:02.940169 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:02.940238 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73,"thread_start_us":49,"threads_started":1}
I20260808 10:08:02.940662 9840 leader_election.cc:288] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c974e52359c14e1aaf0c36f072c48dcc (127.2.30.190:41065), 9f3602e8df67413eaf140373844707a5 (127.2.30.189:46359)
W20260808 10:08:02.940732 9091 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
I20260808 10:08:02.940874 9844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:02.940525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:60174)
0808 10:08:02.940735 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:02.940826 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":90,"threads_started":1}
W20260808 10:08:02.941107 9282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
I20260808 10:08:02.941169 9841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:02.941083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:60180)
0808 10:08:02.941120 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:02.941144 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260808 10:08:02.941246 9091 leader_election.cc:334] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c974e52359c14e1aaf0c36f072c48dcc (127.2.30.190:41065): Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
I20260808 10:08:02.941700 9844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:02.941593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:60182)
0808 10:08:02.941648 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:02.941675 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 10:08:02.941864 9282 leader_election.cc:334] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c974e52359c14e1aaf0c36f072c48dcc (127.2.30.190:41065): Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
I20260808 10:08:02.944417 9842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:02.940108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:39837 (local address 127.0.0.1:56982)
0808 10:08:02.940256 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:02.940266 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:02.940332 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:02.940677 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:02.940679 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:02.940682 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:02.940745 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:02.940748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:02.941701 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:02.941702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:02.943228 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:02.943233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:02.944132 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:02.944134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:02.944209 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:02.944214 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:08:02.944296 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:08:02.944341 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":52,"threads_started":1}
I20260808 10:08:02.944418 9845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:02.940579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:46359 (local address 127.0.0.1:46300)
0808 10:08:02.940882 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:02.940893 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:02.940940 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:02.941158 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:02.941158 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:02.941161 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:02.941195 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:02.941197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:02.942076 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:02.942077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:02.943322 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:02.943326 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:02.944132 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:02.944134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:02.944209 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:02.944214 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:08:02.944288 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 10:08:02.944338 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":53,"threads_started":1}
I20260808 10:08:02.944550 9846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:02.940647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46300 (local address 127.2.30.189:46359)
0808 10:08:02.941020 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:02.941022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:02.941029 (+ 7us) server_negotiation.cc:407] Connection header received
0808 10:08:02.941049 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:02.941049 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:02.941060 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:02.941123 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:02.941262 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:02.942005 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:02.943418 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:02.944010 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:02.944084 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:02.944098 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:02.944352 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 10:08:02.944411 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":40,"threads_started":1}
I20260808 10:08:02.944551 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:02.940401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56982 (local address 127.2.30.188:39837)
0808 10:08:02.940488 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:02.940490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:02.940500 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:08:02.940529 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:02.940530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:02.940541 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:02.940615 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:02.940830 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:02.941632 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:02.943398 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:02.943995 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:02.944085 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:02.944098 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:02.944367 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 10:08:02.944417 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69,"thread_start_us":41,"threads_started":1}
I20260808 10:08:02.944953 9116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b52febb05ad477dac2b5d72e786208c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f3602e8df67413eaf140373844707a5" is_pre_election: true
I20260808 10:08:02.945070 9116 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b52febb05ad477dac2b5d72e786208c in term 1.
I20260808 10:08:02.945113 9307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f3602e8df67413eaf140373844707a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6b52febb05ad477dac2b5d72e786208c" is_pre_election: true
I20260808 10:08:02.945219 9307 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f3602e8df67413eaf140373844707a5 in term 1.
I20260808 10:08:02.945233 9283 leader_election.cc:302] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b52febb05ad477dac2b5d72e786208c, 9f3602e8df67413eaf140373844707a5; no voters: c974e52359c14e1aaf0c36f072c48dcc
I20260808 10:08:02.945348 9840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 10:08:02.945390 9840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 1 FOLLOWER]: Starting leader election (detected failure of leader c974e52359c14e1aaf0c36f072c48dcc)
I20260808 10:08:02.945422 9840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:08:02.945431 9090 leader_election.cc:302] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b52febb05ad477dac2b5d72e786208c, 9f3602e8df67413eaf140373844707a5; no voters: c974e52359c14e1aaf0c36f072c48dcc
I20260808 10:08:02.945541 9839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 10:08:02.945595 9839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c974e52359c14e1aaf0c36f072c48dcc)
I20260808 10:08:02.945611 9839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:08:02.946393 9840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } }
I20260808 10:08:02.946559 9840 leader_election.cc:288] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [CANDIDATE]: Term 2 election: Requested vote from peers c974e52359c14e1aaf0c36f072c48dcc (127.2.30.190:41065), 9f3602e8df67413eaf140373844707a5 (127.2.30.189:46359)
I20260808 10:08:02.946539 9839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } }
I20260808 10:08:02.946702 9839 leader_election.cc:288] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [CANDIDATE]: Term 2 election: Requested vote from peers c974e52359c14e1aaf0c36f072c48dcc (127.2.30.190:41065), 6b52febb05ad477dac2b5d72e786208c (127.2.30.188:39837)
I20260808 10:08:02.946887 9307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9f3602e8df67413eaf140373844707a5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6b52febb05ad477dac2b5d72e786208c"
I20260808 10:08:02.947019 9307 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9f3602e8df67413eaf140373844707a5 in current term 2: Already voted for candidate 6b52febb05ad477dac2b5d72e786208c in this term.
I20260808 10:08:02.946897 9116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b52febb05ad477dac2b5d72e786208c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f3602e8df67413eaf140373844707a5"
I20260808 10:08:02.947090 9845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:02.946718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:60192)
0808 10:08:02.946928 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:02.947009 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192}
I20260808 10:08:02.947172 9116 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b52febb05ad477dac2b5d72e786208c in current term 2: Already voted for candidate 9f3602e8df67413eaf140373844707a5 in this term.
I20260808 10:08:02.947083 9842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:02.946833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:60202)
0808 10:08:02.946875 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:02.946901 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 10:08:02.947475 9845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:02.947324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:60206)
0808 10:08:02.947422 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:02.947449 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260808 10:08:02.947594 9282 leader_election.cc:334] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c974e52359c14e1aaf0c36f072c48dcc (127.2.30.190:41065): Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
I20260808 10:08:02.947641 9842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:02.947554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:60212)
0808 10:08:02.947595 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:02.947620 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 10:08:02.947654 9282 leader_election.cc:302] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b52febb05ad477dac2b5d72e786208c; no voters: 9f3602e8df67413eaf140373844707a5, c974e52359c14e1aaf0c36f072c48dcc
W20260808 10:08:02.947688 9091 leader_election.cc:334] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c974e52359c14e1aaf0c36f072c48dcc (127.2.30.190:41065): Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
I20260808 10:08:02.947727 9091 leader_election.cc:302] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f3602e8df67413eaf140373844707a5; no voters: 6b52febb05ad477dac2b5d72e786208c, c974e52359c14e1aaf0c36f072c48dcc
I20260808 10:08:02.947750 9840 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 10:08:02.947830 9839 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 10:08:03.435822 9849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:03.435518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.131:54477)
0808 10:08:03.435702 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:03.435775 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":109,"threads_started":1}
I20260808 10:08:03.437703 9850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:03.437452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.130:40645)
0808 10:08:03.437616 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:03.437651 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":98,"threads_started":1}
I20260808 10:08:03.437821 9851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:03.437538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.129:56489)
0808 10:08:03.437730 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:03.437777 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":71,"threads_started":1}
I20260808 10:08:04.436583 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.436291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.131:39557)
0808 10:08:04.436483 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.436535 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":96,"threads_started":1}
I20260808 10:08:04.438354 9853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.438156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.130:37575)
0808 10:08:04.438261 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.438300 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":60,"threads_started":1}
I20260808 10:08:04.438686 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.438497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.129:39195)
0808 10:08:04.438602 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.438640 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":48,"threads_started":1}
I20260808 10:08:04.523729 9855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:08:04.523892 9855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } }
I20260808 10:08:04.524228 9855 leader_election.cc:288] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c974e52359c14e1aaf0c36f072c48dcc (127.2.30.190:41065), 9f3602e8df67413eaf140373844707a5 (127.2.30.189:46359)
I20260808 10:08:04.524500 9116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b52febb05ad477dac2b5d72e786208c" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f3602e8df67413eaf140373844707a5" is_pre_election: true
I20260808 10:08:04.524569 9856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.524318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:60386)
0808 10:08:04.524472 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.524530 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":49,"threads_started":1}
I20260808 10:08:04.524626 9116 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b52febb05ad477dac2b5d72e786208c in term 2.
I20260808 10:08:04.524827 9283 leader_election.cc:302] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b52febb05ad477dac2b5d72e786208c, 9f3602e8df67413eaf140373844707a5; no voters:
I20260808 10:08:04.524952 9855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 10:08:04.525010 9855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:08:04.525040 9855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 2 FOLLOWER]: Advancing to term 3
I20260808 10:08:04.525156 9856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.525056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:60400)
0808 10:08:04.525093 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.525121 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260808 10:08:04.525234 9282 leader_election.cc:334] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c974e52359c14e1aaf0c36f072c48dcc (127.2.30.190:41065): Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
I20260808 10:08:04.526158 9855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } }
I20260808 10:08:04.526304 9855 leader_election.cc:288] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [CANDIDATE]: Term 3 election: Requested vote from peers c974e52359c14e1aaf0c36f072c48dcc (127.2.30.190:41065), 9f3602e8df67413eaf140373844707a5 (127.2.30.189:46359)
I20260808 10:08:04.526528 9116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6b52febb05ad477dac2b5d72e786208c" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9f3602e8df67413eaf140373844707a5"
I20260808 10:08:04.526584 9856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.526445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:60406)
0808 10:08:04.526514 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.526559 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 10:08:04.526643 9116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 2 FOLLOWER]: Advancing to term 3
I20260808 10:08:04.527011 9856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.526862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:60412)
0808 10:08:04.526913 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.526941 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 10:08:04.527107 9282 leader_election.cc:334] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c974e52359c14e1aaf0c36f072c48dcc (127.2.30.190:41065): Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
I20260808 10:08:04.527501 9116 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b52febb05ad477dac2b5d72e786208c in term 3.
I20260808 10:08:04.527732 9283 leader_election.cc:302] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b52febb05ad477dac2b5d72e786208c, 9f3602e8df67413eaf140373844707a5; no voters: c974e52359c14e1aaf0c36f072c48dcc
I20260808 10:08:04.527837 9855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 3 FOLLOWER]: Leader election won for term 3
I20260808 10:08:04.527982 9855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [term 3 LEADER]: Becoming Leader. State: Replica: 6b52febb05ad477dac2b5d72e786208c, State: Running, Role: LEADER
I20260808 10:08:04.528051 9855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } }
I20260808 10:08:04.528486 9858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b52febb05ad477dac2b5d72e786208c. Latest consensus state: current_term: 3 leader_uuid: "6b52febb05ad477dac2b5d72e786208c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } } }
I20260808 10:08:04.528604 9858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:04.528798 9860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:08:04.529374 9860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:08:04.529459 9860 catalog_manager.cc:1259] Loaded cluster ID: 8987d372a5fd4b5aa3c86650e67245ab
I20260808 10:08:04.529481 9860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:08:04.529626 9860 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:08:04.529713 9860 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: Loaded TSK: 0
I20260808 10:08:04.529810 9860 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:08:04.556685 9298 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56976:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:08:04.558389 9298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:08:04.559585 9856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.559456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:60428)
0808 10:08:04.559516 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.559547 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 10:08:04.559693 9116 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [term 3 FOLLOWER]: Refusing update from remote peer 6b52febb05ad477dac2b5d72e786208c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 10:08:04.560048 9856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.559907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.0.0.1:60440)
0808 10:08:04.559957 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.559986 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 10:08:04.559984 9858 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 10:08:04.560276 9282 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c -> Peer c974e52359c14e1aaf0c36f072c48dcc (127.2.30.190:41065): Couldn't send request to peer c974e52359c14e1aaf0c36f072c48dcc. Status: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 10:08:04.560819 9862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b52febb05ad477dac2b5d72e786208c. Latest consensus state: current_term: 3 leader_uuid: "6b52febb05ad477dac2b5d72e786208c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } } }
I20260808 10:08:04.560921 9862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:08:04.561435 9855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "6b52febb05ad477dac2b5d72e786208c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } } }
I20260808 10:08:04.561699 9863 mvcc.cc:205] Tried to move back new op lower bound from 7316208371953655808 to 7316208371827699712. Current Snapshot: MvccSnapshot[applied={T|T < 7316208371953655808}]
I20260808 10:08:04.561697 9862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "6b52febb05ad477dac2b5d72e786208c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c974e52359c14e1aaf0c36f072c48dcc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41065 } } peers { permanent_uuid: "9f3602e8df67413eaf140373844707a5" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 46359 } attrs { promote: false } } peers { permanent_uuid: "6b52febb05ad477dac2b5d72e786208c" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 39837 } attrs { promote: false } } }
I20260808 10:08:04.561892 9862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f3602e8df67413eaf140373844707a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:08:04.561573 9855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:04.567102 9856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.563355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:33039 (local address 127.0.0.1:45072)
0808 10:08:04.563435 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.563445 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:04.563495 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:04.563769 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:04.563771 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:04.563773 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:04.563858 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:04.563860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:04.565089 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:04.565091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:04.566151 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:04.566153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:04.566773 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:04.566774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:04.566848 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:04.566852 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:08:04.566946 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 10:08:04.567019 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":47}
I20260808 10:08:04.567104 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.563432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45072 (local address 127.2.30.131:33039)
0808 10:08:04.563567 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:04.563570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:04.563580 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:08:04.563612 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:04.563612 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:04.563626 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:04.563959 (+ 333us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:04.564009 (+ 50us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:04.564984 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:04.566225 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:04.566655 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:04.566770 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:04.566784 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:04.566956 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 10:08:04.567026 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260808 10:08:04.567441 9698 tablet_service.cc:1467] Processing CreateTablet for tablet 2df32df69fa74bddb347b10f795173ef (DEFAULT_TABLE table=test-table [id=d0c58dbb7a4647c39cb52640b32f5e09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:04.567591 9698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2df32df69fa74bddb347b10f795173ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:04.568301 9874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.563714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42228 (local address 127.2.30.130:36601)
0808 10:08:04.565137 (+ 1423us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:04.565140 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:04.565148 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:08:04.565188 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:04.565189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:04.565198 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:04.565262 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:04.565397 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:04.565941 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:04.567307 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:04.567899 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:04.567979 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:04.567991 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:04.568177 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 10:08:04.568224 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1398,"thread_start_us":75,"threads_started":1}
I20260808 10:08:04.568387 9871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.563479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:36601 (local address 127.0.0.1:42228)
0808 10:08:04.563740 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.563750 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:04.563825 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:04.565264 (+ 1439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:04.565265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:04.565267 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:04.565327 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:04.565329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:04.566031 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:04.566032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:04.567214 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:04.567218 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:04.568005 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:04.568007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:04.568080 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:04.568085 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:08:04.568161 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:08:04.568272 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":19,"thread_start_us":120,"threads_started":1}
I20260808 10:08:04.568665 9870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.563442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:44221 (local address 127.0.0.1:45582)
0808 10:08:04.563716 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.563727 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:04.563776 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:04.564057 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:04.564058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:04.564060 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:04.564105 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:04.564107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:04.565089 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:04.565091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:04.567407 (+ 2316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:04.567410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:04.568416 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:04.568417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:04.568489 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:04.568494 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:08:04.568551 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 10:08:04.568598 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":20,"thread_start_us":78,"threads_started":1}
I20260808 10:08:04.568640 9556 tablet_service.cc:1467] Processing CreateTablet for tablet 2df32df69fa74bddb347b10f795173ef (DEFAULT_TABLE table=test-table [id=d0c58dbb7a4647c39cb52640b32f5e09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:04.568715 9872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.563639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45582 (local address 127.2.30.129:44221)
0808 10:08:04.563914 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:04.563916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:04.563923 (+ 7us) server_negotiation.cc:407] Connection header received
0808 10:08:04.563948 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:04.563949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:04.563959 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:04.564039 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:04.564197 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:04.565003 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:04.567494 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:04.568097 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:04.568190 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:04.568201 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:04.568578 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 10:08:04.568634 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":49,"threads_started":1}
I20260808 10:08:04.568774 9556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2df32df69fa74bddb347b10f795173ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:04.569205 9875 tablet_bootstrap.cc:492] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e: Bootstrap starting.
I20260808 10:08:04.569662 9875 tablet_bootstrap.cc:654] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:04.570241 9877 tablet_bootstrap.cc:492] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf: Bootstrap starting.
I20260808 10:08:04.570398 9415 tablet_service.cc:1467] Processing CreateTablet for tablet 2df32df69fa74bddb347b10f795173ef (DEFAULT_TABLE table=test-table [id=d0c58dbb7a4647c39cb52640b32f5e09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:04.570433 9875 tablet_bootstrap.cc:492] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e: No bootstrap required, opened a new log
I20260808 10:08:04.570483 9875 ts_tablet_manager.cc:1397] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:04.570523 9415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2df32df69fa74bddb347b10f795173ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:04.570608 9875 raft_consensus.cc:348] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:04.570675 9875 raft_consensus.cc:374] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:04.570690 9875 raft_consensus.cc:729] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71379382edcc4184aa937dbb8098251e, State: Initialized, Role: FOLLOWER
I20260808 10:08:04.570727 9875 consensus_queue.cc:260] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:04.570924 9875 ts_tablet_manager.cc:1428] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:08:04.571331 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.571192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.131:33073)
0808 10:08:04.571266 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.571303 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 10:08:04.571722 9767 heartbeater.cc:503] Master 127.2.30.188:39837 was elected leader, sending a full tablet report...
I20260808 10:08:04.571960 9879 tablet_bootstrap.cc:492] T 2df32df69fa74bddb347b10f795173ef P 74d1601b5a52457a951fafd40b699521: Bootstrap starting.
I20260808 10:08:04.572388 9879 tablet_bootstrap.cc:654] T 2df32df69fa74bddb347b10f795173ef P 74d1601b5a52457a951fafd40b699521: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:04.572957 9877 tablet_bootstrap.cc:654] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:04.573062 9879 tablet_bootstrap.cc:492] T 2df32df69fa74bddb347b10f795173ef P 74d1601b5a52457a951fafd40b699521: No bootstrap required, opened a new log
I20260808 10:08:04.573119 9879 ts_tablet_manager.cc:1397] T 2df32df69fa74bddb347b10f795173ef P 74d1601b5a52457a951fafd40b699521: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:04.573302 9879 raft_consensus.cc:348] T 2df32df69fa74bddb347b10f795173ef P 74d1601b5a52457a951fafd40b699521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:04.573410 9879 raft_consensus.cc:374] T 2df32df69fa74bddb347b10f795173ef P 74d1601b5a52457a951fafd40b699521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:04.573433 9879 raft_consensus.cc:729] T 2df32df69fa74bddb347b10f795173ef P 74d1601b5a52457a951fafd40b699521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74d1601b5a52457a951fafd40b699521, State: Initialized, Role: FOLLOWER
I20260808 10:08:04.573487 9879 consensus_queue.cc:260] T 2df32df69fa74bddb347b10f795173ef P 74d1601b5a52457a951fafd40b699521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:04.573673 9879 ts_tablet_manager.cc:1428] T 2df32df69fa74bddb347b10f795173ef P 74d1601b5a52457a951fafd40b699521: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:04.573985 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.573865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.129:57769)
0808 10:08:04.573924 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.573956 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 10:08:04.574237 9877 tablet_bootstrap.cc:492] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf: No bootstrap required, opened a new log
I20260808 10:08:04.574304 9483 heartbeater.cc:503] Master 127.2.30.188:39837 was elected leader, sending a full tablet report...
I20260808 10:08:04.574329 9877 ts_tablet_manager.cc:1397] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 10:08:04.574415 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.574312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.129:38803)
0808 10:08:04.574352 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.574390 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 10:08:04.574455 9877 raft_consensus.cc:348] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:04.574536 9877 raft_consensus.cc:374] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:04.574568 9877 raft_consensus.cc:729] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a116392f87ac403ab0d65007de2db5cf, State: Initialized, Role: FOLLOWER
I20260808 10:08:04.574622 9877 consensus_queue.cc:260] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:04.574810 9877 ts_tablet_manager.cc:1428] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:04.575150 9853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.575039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.130:47963)
0808 10:08:04.575081 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.575119 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 10:08:04.575436 9626 heartbeater.cc:503] Master 127.2.30.188:39837 was elected leader, sending a full tablet report...
I20260808 10:08:04.575598 9853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.575496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.130:43147)
0808 10:08:04.575533 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.575564 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 10:08:04.827473 9883 raft_consensus.cc:482] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:08:04.827621 9883 raft_consensus.cc:504] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:04.827877 9883 leader_election.cc:288] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71379382edcc4184aa937dbb8098251e (127.2.30.131:33039), 74d1601b5a52457a951fafd40b699521 (127.2.30.129:44221)
I20260808 10:08:04.828207 9719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2df32df69fa74bddb347b10f795173ef" candidate_uuid: "a116392f87ac403ab0d65007de2db5cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71379382edcc4184aa937dbb8098251e" is_pre_election: true
I20260808 10:08:04.828334 9719 raft_consensus.cc:2454] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a116392f87ac403ab0d65007de2db5cf in term 0.
I20260808 10:08:04.828291 9436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2df32df69fa74bddb347b10f795173ef" candidate_uuid: "a116392f87ac403ab0d65007de2db5cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74d1601b5a52457a951fafd40b699521" is_pre_election: true
I20260808 10:08:04.828382 9436 raft_consensus.cc:2454] T 2df32df69fa74bddb347b10f795173ef P 74d1601b5a52457a951fafd40b699521 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a116392f87ac403ab0d65007de2db5cf in term 0.
I20260808 10:08:04.828526 9509 leader_election.cc:302] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71379382edcc4184aa937dbb8098251e, a116392f87ac403ab0d65007de2db5cf; no voters:
I20260808 10:08:04.828681 9883 raft_consensus.cc:2781] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:08:04.828727 9883 raft_consensus.cc:482] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:08:04.828747 9883 raft_consensus.cc:3037] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:04.829569 9883 raft_consensus.cc:504] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:04.829736 9883 leader_election.cc:288] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [CANDIDATE]: Term 1 election: Requested vote from peers 71379382edcc4184aa937dbb8098251e (127.2.30.131:33039), 74d1601b5a52457a951fafd40b699521 (127.2.30.129:44221)
I20260808 10:08:04.829939 9436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2df32df69fa74bddb347b10f795173ef" candidate_uuid: "a116392f87ac403ab0d65007de2db5cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74d1601b5a52457a951fafd40b699521"
I20260808 10:08:04.830022 9436 raft_consensus.cc:3037] T 2df32df69fa74bddb347b10f795173ef P 74d1601b5a52457a951fafd40b699521 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:04.829988 9719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2df32df69fa74bddb347b10f795173ef" candidate_uuid: "a116392f87ac403ab0d65007de2db5cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71379382edcc4184aa937dbb8098251e"
I20260808 10:08:04.830107 9719 raft_consensus.cc:3037] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:04.830744 9719 raft_consensus.cc:2454] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a116392f87ac403ab0d65007de2db5cf in term 1.
I20260808 10:08:04.830756 9436 raft_consensus.cc:2454] T 2df32df69fa74bddb347b10f795173ef P 74d1601b5a52457a951fafd40b699521 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a116392f87ac403ab0d65007de2db5cf in term 1.
I20260808 10:08:04.830940 9509 leader_election.cc:302] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71379382edcc4184aa937dbb8098251e, a116392f87ac403ab0d65007de2db5cf; no voters:
I20260808 10:08:04.831113 9883 raft_consensus.cc:2781] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:08:04.831274 9883 raft_consensus.cc:686] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [term 1 LEADER]: Becoming Leader. State: Replica: a116392f87ac403ab0d65007de2db5cf, State: Running, Role: LEADER
I20260808 10:08:04.831364 9883 consensus_queue.cc:237] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } } peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } }
I20260808 10:08:04.831686 9853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.831571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.130:51903)
0808 10:08:04.831614 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.831658 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260808 10:08:04.832125 9298 catalog_manager.cc:5495] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf reported cstate change: term changed from 0 to 1, leader changed from <none> to a116392f87ac403ab0d65007de2db5cf (127.2.30.130). New cstate: current_term: 1 leader_uuid: "a116392f87ac403ab0d65007de2db5cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a116392f87ac403ab0d65007de2db5cf" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36601 } health_report { overall_health: HEALTHY } } }
I20260808 10:08:04.853134 9874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.850649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42238 (local address 127.2.30.130:36601)
0808 10:08:04.850725 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:04.850727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:04.850736 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:08:04.850826 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:04.850827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:04.850839 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:04.850903 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:08:04.851085 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:04.851620 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:04.852380 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:04.852588 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:04.852670 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:04.852840 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:08:04.852878 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:08:04.852939 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:04.853004 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 10:08:04.853028 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:08:04.854218 9719 raft_consensus.cc:1261] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e [term 1 FOLLOWER]: Refusing update from remote peer a116392f87ac403ab0d65007de2db5cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:08:04.854271 9436 raft_consensus.cc:1261] T 2df32df69fa74bddb347b10f795173ef P 74d1601b5a52457a951fafd40b699521 [term 1 FOLLOWER]: Refusing update from remote peer a116392f87ac403ab0d65007de2db5cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:08:04.854521 9883 consensus_queue.cc:1035] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "71379382edcc4184aa937dbb8098251e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 33039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:08:04.854617 9884 consensus_queue.cc:1035] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "74d1601b5a52457a951fafd40b699521" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:08:04.855688 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.855316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.131:40503)
0808 10:08:04.855613 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.855653 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273}
I20260808 10:08:04.855996 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.855559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.129:45393)
0808 10:08:04.855925 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.855959 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346}
I20260808 10:08:04.856427 9853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.856289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.130:41243)
0808 10:08:04.856360 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.856395 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 10:08:04.857230 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.856786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.129:49491)
0808 10:08:04.857168 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.857198 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365}
I20260808 10:08:04.858283 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.858192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.131:60597)
0808 10:08:04.858230 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.858256 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 10:08:04.858418 9853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.858277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41065 (local address 127.2.30.130:36747)
0808 10:08:04.858348 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.858380 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:41065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 10:08:04.858946 9297 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:08:04.859066 9297 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:08:04.860571 9297 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b52febb05ad477dac2b5d72e786208c: Sending DeleteTablet for 3 replicas of tablet 2df32df69fa74bddb347b10f795173ef
I20260808 10:08:04.860946 9698 tablet_service.cc:1514] Processing DeleteTablet for tablet 2df32df69fa74bddb347b10f795173ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:04 UTC) from {username='oryx', principal='oryx/127.2.30.188@KRBTEST.COM'} at 127.0.0.1:45072
I20260808 10:08:04.860975 9415 tablet_service.cc:1514] Processing DeleteTablet for tablet 2df32df69fa74bddb347b10f795173ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:04 UTC) from {username='oryx', principal='oryx/127.2.30.188@KRBTEST.COM'} at 127.0.0.1:45582
I20260808 10:08:04.861155 9556 tablet_service.cc:1514] Processing DeleteTablet for tablet 2df32df69fa74bddb347b10f795173ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:04 UTC) from {username='oryx', principal='oryx/127.2.30.188@KRBTEST.COM'} at 127.0.0.1:42228
I20260808 10:08:04.861308 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 9256
I20260808 10:08:04.862645 9897 tablet_replica.cc:330] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf: stopping tablet replica
I20260808 10:08:04.862725 9897 raft_consensus.cc:2229] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:08:04.862803 9897 raft_consensus.cc:2258] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:04.863410 9897 ts_tablet_manager.cc:1905] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:04.865132 9897 ts_tablet_manager.cc:1918] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:08:04.865181 9897 log.cc:1196] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/2df32df69fa74bddb347b10f795173ef
I20260808 10:08:04.865408 9897 ts_tablet_manager.cc:1939] T 2df32df69fa74bddb347b10f795173ef P a116392f87ac403ab0d65007de2db5cf: Deleting consensus metadata
I20260808 10:08:04.865720 9284 catalog_manager.cc:4841] TS a116392f87ac403ab0d65007de2db5cf (127.2.30.130:36601): tablet 2df32df69fa74bddb347b10f795173ef (table test-table [id=d0c58dbb7a4647c39cb52640b32f5e09]) successfully deleted
I20260808 10:08:04.865777 9898 tablet_replica.cc:330] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e: stopping tablet replica
I20260808 10:08:04.865837 9898 raft_consensus.cc:2229] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:08:04.865904 9898 raft_consensus.cc:2258] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:04.866217 9898 ts_tablet_manager.cc:1905] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:04.867592 9898 ts_tablet_manager.cc:1918] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:08:04.867674 9898 log.cc:1196] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/2df32df69fa74bddb347b10f795173ef
I20260808 10:08:04.867921 9898 ts_tablet_manager.cc:1939] T 2df32df69fa74bddb347b10f795173ef P 71379382edcc4184aa937dbb8098251e: Deleting consensus metadata
I20260808 10:08:04.868423 9282 catalog_manager.cc:4841] TS 71379382edcc4184aa937dbb8098251e (127.2.30.131:33039): tablet 2df32df69fa74bddb347b10f795173ef (table test-table [id=d0c58dbb7a4647c39cb52640b32f5e09]) successfully deleted
I20260808 10:08:04.871018 9870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:04.870822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:44221 (local address 127.0.0.1:45596)
0808 10:08:04.870918 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:04.870948 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:44221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 10:08:04.871155 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 9491
W20260808 10:08:04.874689 9281 catalog_manager.cc:4584] TS 74d1601b5a52457a951fafd40b699521 (127.2.30.129:44221): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2df32df69fa74bddb347b10f795173ef: Network error: Client connection negotiation failed: client connection to 127.2.30.129:44221: connect: Connection refused (error 111)
I20260808 10:08:04.880487 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 9633
I20260808 10:08:04.886149 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 8967
I20260808 10:08:04.892737 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 9160
2026-08-08T10:08:04Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6722 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:08:04 dist-test-slave-w206 krb5kdc[9901](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:08:04 dist-test-slave-w206 krb5kdc[9901](info): set up 2 sockets
Aug 08 10:08:04 dist-test-slave-w206 krb5kdc[9901](info): commencing operation
krb5kdc: starting...
W20260808 10:08:05.972273 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:08:06 dist-test-slave-w206 krb5kdc[9901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:08:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:08:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.190.keytab.
Entry for principal oryx/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.190.keytab.
I20260808 10:08:06.111317 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:46135
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.190.keytab
--principal=oryx/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:42171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:46135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:06.196998 9914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:06.197253 9914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:06.197294 9914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:06.198491 9914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:08:06.198565 9914 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 10:08:06.198587 9914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:06.198611 9914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:06.198661 9914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:08:06.198691 9914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:08:06.200371 9914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:42171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:46135
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.190.keytab
--principal=oryx/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:46135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:06.200657 9914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:06.200901 9914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:06.206260 9920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:06.206354 9914 server_base.cc:1034] running on GCE node
W20260808 10:08:06.206322 9922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:06.206290 9919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:06.206869 9914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:06.207206 9914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:06.208380 9914 hybrid_clock.cc:648] HybridClock initialized: now 1786183686208364 us; error 65 us; skew 500 ppm
Aug 08 10:08:06 dist-test-slave-w206 krb5kdc[9901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183686, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:06.211195 9914 init.cc:377] Logged in from keytab as oryx/127.2.30.190@KRBTEST.COM (short username oryx)
I20260808 10:08:06.211710 9914 webserver.cc:460] Webserver started at http://127.2.30.190:44459/ using document root <none> and password file <none>
I20260808 10:08:06.211910 9914 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:06.211952 9914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:06.212050 9914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:06.213229 9914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "c34ab8d460aa4d8e9d5ab3dec6e36a94"
format_stamp: "Formatted at 2026-08-08 10:08:06 on dist-test-slave-w206"
server_key: "8dcc96422405f50082b3cd3f0d281a04"
server_key_iv: "689a4b3c0c76e79f627462ebfe6edd4c"
server_key_version: "encryptionkey@0"
I20260808 10:08:06.213557 9914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "c34ab8d460aa4d8e9d5ab3dec6e36a94"
format_stamp: "Formatted at 2026-08-08 10:08:06 on dist-test-slave-w206"
server_key: "8dcc96422405f50082b3cd3f0d281a04"
server_key_iv: "689a4b3c0c76e79f627462ebfe6edd4c"
server_key_version: "encryptionkey@0"
I20260808 10:08:06.215412 9914 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:08:06.216279 9930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:06.216544 9914 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:06.216650 9914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "c34ab8d460aa4d8e9d5ab3dec6e36a94"
format_stamp: "Formatted at 2026-08-08 10:08:06 on dist-test-slave-w206"
server_key: "8dcc96422405f50082b3cd3f0d281a04"
server_key_iv: "689a4b3c0c76e79f627462ebfe6edd4c"
server_key_version: "encryptionkey@0"
I20260808 10:08:06.216725 9914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:06.241031 9914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:06.243350 9914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:06.243512 9914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:06.249011 9914 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:46135
I20260808 10:08:06.249128 9995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:46135 every 8 connection(s)
I20260808 10:08:06.249536 9914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:08:06.250422 9996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:06.253024 9996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94: Bootstrap starting.
I20260808 10:08:06.253742 9996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:06.254060 9996 log.cc:824] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:06.254923 9996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94: No bootstrap required, opened a new log
I20260808 10:08:06.257225 9996 raft_consensus.cc:348] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34ab8d460aa4d8e9d5ab3dec6e36a94" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46135 } }
I20260808 10:08:06.257386 9996 raft_consensus.cc:374] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:06.257417 9996 raft_consensus.cc:729] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c34ab8d460aa4d8e9d5ab3dec6e36a94, State: Initialized, Role: FOLLOWER
I20260808 10:08:06.257624 9996 consensus_queue.cc:260] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34ab8d460aa4d8e9d5ab3dec6e36a94" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46135 } }
I20260808 10:08:06.257716 9996 raft_consensus.cc:388] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:08:06.257733 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 9914
I20260808 10:08:06.257742 9996 raft_consensus.cc:482] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:08:06.257810 9996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:06.257840 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:08:06.258034 2170 external_mini_cluster.cc:1428] Setting key a7e6bc680e2fdf2aa899e7152702302e
I20260808 10:08:06.258664 9996 raft_consensus.cc:504] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34ab8d460aa4d8e9d5ab3dec6e36a94" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46135 } }
I20260808 10:08:06.258852 9996 leader_election.cc:302] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c34ab8d460aa4d8e9d5ab3dec6e36a94; no voters:
I20260808 10:08:06.259068 9996 leader_election.cc:288] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:08:06.259104 10001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:08:06.259210 10001 raft_consensus.cc:686] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [term 1 LEADER]: Becoming Leader. State: Replica: c34ab8d460aa4d8e9d5ab3dec6e36a94, State: Running, Role: LEADER
I20260808 10:08:06.259311 10001 consensus_queue.cc:237] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34ab8d460aa4d8e9d5ab3dec6e36a94" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46135 } }
I20260808 10:08:06.259370 9996 sys_catalog.cc:564] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:08:06.259646 10001 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c34ab8d460aa4d8e9d5ab3dec6e36a94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34ab8d460aa4d8e9d5ab3dec6e36a94" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46135 } } }
I20260808 10:08:06.259732 10001 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:06.259862 10001 sys_catalog.cc:455] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c34ab8d460aa4d8e9d5ab3dec6e36a94. Latest consensus state: current_term: 1 leader_uuid: "c34ab8d460aa4d8e9d5ab3dec6e36a94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c34ab8d460aa4d8e9d5ab3dec6e36a94" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46135 } } }
I20260808 10:08:06.259950 10001 sys_catalog.cc:458] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94 [sys.catalog]: This master's current role is: LEADER
W20260808 10:08:06.261318 10019 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:08:06.261380 10019 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:08:06.261428 10006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:08:06.261566 10006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:08:06.263283 10006 catalog_manager.cc:1347] Generated new cluster ID: bb7bf423ad074ceea2490fa26e7a0470
I20260808 10:08:06.263347 10006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:08:06 dist-test-slave-w206 krb5kdc[9901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM
I20260808 10:08:06.269896 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.258620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44214 (local address 127.2.30.190:46135)
0808 10:08:06.258883 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:06.258892 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:06.258927 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:08:06.259091 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:06.259097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:06.259177 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:06.259386 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:06.260440 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.261722 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.262696 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.263386 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.263492 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.266446 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:06.266450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:06.266451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:06.266464 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:06.268278 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:06.268683 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:06.268692 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:06.268692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:06.268797 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:06.269017 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:06.269017 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:06.269018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:06.269278 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:06.269461 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:06.269563 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 10:08:06.269717 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":127,"threads_started":1}
I20260808 10:08:06.270757 10006 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:08:06.271476 10006 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:08:06.279512 10006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94: Generated new TSK 0
I20260808 10:08:06.279748 10006 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.129.keytab.
Entry for principal oryx/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.129.keytab.
I20260808 10:08:06.314101 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46135
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.129.keytab
--principal=oryx/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:42171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:06.401943 10024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:06.402266 10024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:06.402310 10024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:06.403561 10024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:06.403605 10024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:06.403626 10024 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:08:06.403644 10024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:08:06.405130 10024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:42171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.129.keytab
--principal=oryx/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.30.190:46135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:06.405331 10024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:06.405515 10024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:06.410964 10033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:06.411028 10031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:06.410965 10030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:06.411373 10024 server_base.cc:1034] running on GCE node
I20260808 10:08:06.411567 10024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:06.411826 10024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:06.412984 10024 hybrid_clock.cc:648] HybridClock initialized: now 1786183686412973 us; error 32 us; skew 500 ppm
Aug 08 10:08:06 dist-test-slave-w206 krb5kdc[9901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183686, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:06.415745 10024 init.cc:377] Logged in from keytab as oryx/127.2.30.129@KRBTEST.COM (short username oryx)
I20260808 10:08:06.416235 10024 webserver.cc:460] Webserver started at http://127.2.30.129:34731/ using document root <none> and password file <none>
I20260808 10:08:06.416453 10024 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:06.416494 10024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:06.416607 10024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:06.417727 10024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "b4c570f847b34048861611cc7eb2bc5b"
format_stamp: "Formatted at 2026-08-08 10:08:06 on dist-test-slave-w206"
server_key: "6579acc769f0d8bee14cb53d4da270a1"
server_key_iv: "22a954119a7055724edb158d61a049ae"
server_key_version: "encryptionkey@0"
I20260808 10:08:06.418105 10024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "b4c570f847b34048861611cc7eb2bc5b"
format_stamp: "Formatted at 2026-08-08 10:08:06 on dist-test-slave-w206"
server_key: "6579acc769f0d8bee14cb53d4da270a1"
server_key_iv: "22a954119a7055724edb158d61a049ae"
server_key_version: "encryptionkey@0"
I20260808 10:08:06.419610 10024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:08:06.420517 10040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:06.420717 10024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:08:06.420817 10024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "b4c570f847b34048861611cc7eb2bc5b"
format_stamp: "Formatted at 2026-08-08 10:08:06 on dist-test-slave-w206"
server_key: "6579acc769f0d8bee14cb53d4da270a1"
server_key_iv: "22a954119a7055724edb158d61a049ae"
server_key_version: "encryptionkey@0"
I20260808 10:08:06.420920 10024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:06.437263 10024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:06.439980 10024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:06.440160 10024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:06.440876 10024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:06.440932 10024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:06.440970 10024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:06.440994 10024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:08:06 dist-test-slave-w206 krb5kdc[9901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183686, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM
I20260808 10:08:06.449081 10024 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:45649
I20260808 10:08:06.449311 10157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:45649 every 8 connection(s)
I20260808 10:08:06.449572 10024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:08:06.450882 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 10024
I20260808 10:08:06.451047 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:08:06.451298 2170 external_mini_cluster.cc:1428] Setting key 4f5386ed43daf294cb669f1767885a8b
I20260808 10:08:06.451975 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.441378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:52429 (local address 127.2.30.190:46135)
0808 10:08:06.441462 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:06.441466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:06.441698 (+ 232us) server_negotiation.cc:407] Connection header received
0808 10:08:06.442239 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:06.442241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:06.442259 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:06.442341 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:06.443214 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.443777 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.444879 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.445087 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.445178 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.448296 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:06.448298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:06.448298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:06.448308 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:06.450296 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:06.450790 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:06.450791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:06.450791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:06.450858 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:06.451201 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:06.451202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:06.451202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:06.451371 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:06.451447 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:06.451785 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 10:08:06.451884 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:08:06.452180 10049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.441372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46135 (local address 127.2.30.129:52429)
0808 10:08:06.441555 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:06.441599 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:06.442134 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:06.442436 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:06.442440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:06.442791 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:06.443104 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.443109 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.443881 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.443883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.444747 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.444750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.445189 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.445190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.445332 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.445935 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:06.445953 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:06.448097 (+ 2144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:06.450407 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:06.450415 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:06.450418 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:06.450711 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:06.450942 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:06.450943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:06.450943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:06.451121 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:06.451437 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:06.451445 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:06.451634 (+ 189us) client_negotiation.cc:769] Sending connection context
0808 10:08:06.451739 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 10:08:06.451921 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
I20260808 10:08:06.452749 10158 heartbeater.cc:347] Connected to a master server at 127.2.30.190:46135
I20260808 10:08:06.452862 10158 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:06.453059 10158 heartbeater.cc:511] Master 127.2.30.190:46135 requested a full tablet report, sending...
I20260808 10:08:06.453666 9947 ts_manager.cc:194] Registered new tserver with Master: b4c570f847b34048861611cc7eb2bc5b (127.2.30.129:45649)
I20260808 10:08:06.454325 9947 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:52429
I20260808 10:08:06.461714 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.454518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:36187 (local address 127.2.30.190:46135)
0808 10:08:06.454557 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:06.454560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:06.455165 (+ 605us) server_negotiation.cc:407] Connection header received
0808 10:08:06.455255 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:06.455256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:06.455266 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:06.455319 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:06.455622 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.456126 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.457345 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.457537 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.457602 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.458859 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:06.458860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:06.458860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:06.458866 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:06.460532 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:06.460954 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:06.460955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:06.460955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:06.461019 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:06.461238 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:06.461239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:06.461239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:06.461343 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:06.461394 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:06.461582 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 10:08:06.461645 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 10:08:06.461804 10167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.454525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46135 (local address 127.2.30.129:36187)
0808 10:08:06.455088 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:06.455101 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:06.455199 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:06.455326 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:06.455326 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:06.455507 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:06.455558 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.455560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.456234 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.456235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.457151 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.457154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.457625 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.457627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.457708 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.458244 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:06.458250 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:06.458723 (+ 473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:06.460613 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:06.460615 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:06.460616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:06.460854 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:06.461089 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:06.461090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:06.461090 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:06.461169 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:06.461415 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:06.461416 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:06.461477 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 10:08:06.461568 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 10:08:06.461687 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for oryx/127.2.30.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.130.keytab.
Entry for principal oryx/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.130.keytab.
I20260808 10:08:06.482755 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46135
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.130.keytab
--principal=oryx/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:42171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:06.567322 10169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:06.567576 10169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:06.567618 10169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:06.568817 10169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:06.568885 10169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:06.568920 10169 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:08:06.568964 10169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:08:06.570556 10169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:42171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.130.keytab
--principal=oryx/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.30.190:46135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:06.570853 10169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:06.571131 10169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:06.576622 10175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:06.576622 10174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:06.576712 10177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:06.576982 10169 server_base.cc:1034] running on GCE node
I20260808 10:08:06.577234 10169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:06.577517 10169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:06.578666 10169 hybrid_clock.cc:648] HybridClock initialized: now 1786183686578652 us; error 28 us; skew 500 ppm
Aug 08 10:08:06 dist-test-slave-w206 krb5kdc[9901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183686, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:06.581405 10169 init.cc:377] Logged in from keytab as oryx/127.2.30.130@KRBTEST.COM (short username oryx)
I20260808 10:08:06.581925 10169 webserver.cc:460] Webserver started at http://127.2.30.130:43855/ using document root <none> and password file <none>
I20260808 10:08:06.582183 10169 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:06.582232 10169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:06.582423 10169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:06.583739 10169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "efbe6fa7b3704e8a84a2638603210731"
format_stamp: "Formatted at 2026-08-08 10:08:06 on dist-test-slave-w206"
server_key: "13e5ce6c5db609718313f13848959ad3"
server_key_iv: "42cfeef310111b3849a5a42e96ecd319"
server_key_version: "encryptionkey@0"
I20260808 10:08:06.584110 10169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "efbe6fa7b3704e8a84a2638603210731"
format_stamp: "Formatted at 2026-08-08 10:08:06 on dist-test-slave-w206"
server_key: "13e5ce6c5db609718313f13848959ad3"
server_key_iv: "42cfeef310111b3849a5a42e96ecd319"
server_key_version: "encryptionkey@0"
I20260808 10:08:06.585515 10169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:08:06.586344 10184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:06.586557 10169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:06.586668 10169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "efbe6fa7b3704e8a84a2638603210731"
format_stamp: "Formatted at 2026-08-08 10:08:06 on dist-test-slave-w206"
server_key: "13e5ce6c5db609718313f13848959ad3"
server_key_iv: "42cfeef310111b3849a5a42e96ecd319"
server_key_version: "encryptionkey@0"
I20260808 10:08:06.586756 10169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:06.600880 10169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:06.603499 10169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:06.603678 10169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:06.604336 10169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:06.604408 10169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:06.604451 10169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:06.604475 10169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 10:08:06 dist-test-slave-w206 krb5kdc[9901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183686, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM
I20260808 10:08:06.611719 10169 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:41569
I20260808 10:08:06.611809 10303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:41569 every 8 connection(s)
I20260808 10:08:06.612106 10169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:08:06.614400 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.604776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:44125 (local address 127.2.30.190:46135)
0808 10:08:06.604848 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:06.604851 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:06.605075 (+ 224us) server_negotiation.cc:407] Connection header received
0808 10:08:06.605603 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:06.605604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:06.605616 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:06.605674 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:06.606542 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.607219 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.608237 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.608451 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.608530 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.611457 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:06.611458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:06.611459 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:06.611467 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:06.612970 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:06.613412 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:06.613413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:06.613413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:06.613478 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:06.613733 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:06.613734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:06.613734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:06.613857 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:06.613928 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:06.614211 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 10:08:06.614306 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:08:06.614591 10193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.604764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46135 (local address 127.2.30.130:44125)
0808 10:08:06.604940 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:06.604981 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:06.605496 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:06.605753 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:06.605757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:06.606115 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:06.606437 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.606443 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.607320 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.607321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.608144 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.608147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.608537 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.608538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.608680 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.609278 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:06.609297 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:06.611351 (+ 2054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:06.613066 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:06.613073 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:06.613076 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:06.613331 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:06.613553 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:06.613553 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:06.613554 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:06.613663 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:06.613930 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:06.613938 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:06.614076 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 10:08:06.614187 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 10:08:06.614354 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":110,"threads_started":1}
I20260808 10:08:06.615197 10304 heartbeater.cc:347] Connected to a master server at 127.2.30.190:46135
I20260808 10:08:06.615314 10304 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:06.615513 10304 heartbeater.cc:511] Master 127.2.30.190:46135 requested a full tablet report, sending...
I20260808 10:08:06.615936 9948 ts_manager.cc:194] Registered new tserver with Master: efbe6fa7b3704e8a84a2638603210731 (127.2.30.130:41569)
I20260808 10:08:06.616519 9948 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:44125
I20260808 10:08:06.619190 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 10169
I20260808 10:08:06.619372 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:08:06.619597 2170 external_mini_cluster.cc:1428] Setting key 39cfe446779c235ba939db1262bfb0f9
I20260808 10:08:06.622866 10310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.616254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46135 (local address 127.2.30.130:33681)
0808 10:08:06.616413 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:06.616422 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:06.616527 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:06.616656 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:06.616657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:06.616834 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:06.616921 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.616923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.617677 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.617678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.618173 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.618174 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.618588 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.618589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.618659 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.619299 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:06.619304 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:06.619768 (+ 464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:06.621585 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:06.621586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:06.621587 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:06.621878 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:06.622089 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:06.622090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:06.622090 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:06.622175 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:06.622558 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:06.622559 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:06.622605 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 10:08:06.622681 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:08:06.622779 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":64,"threads_started":1}
I20260808 10:08:06.622972 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.616326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:33681 (local address 127.2.30.190:46135)
0808 10:08:06.616371 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:06.616373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:06.616491 (+ 118us) server_negotiation.cc:407] Connection header received
0808 10:08:06.616587 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:06.616588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:06.616597 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:06.616653 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:06.616999 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.617579 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.618289 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.618486 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.618550 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.619910 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:06.619912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:06.619912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:06.619921 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:06.621496 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:06.621963 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:06.621964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:06.621964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:06.622016 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:06.622254 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:06.622255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:06.622256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:06.622473 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:06.622549 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:06.622729 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 10:08:06.622831 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for oryx/127.2.30.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.131.keytab.
Entry for principal oryx/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.131.keytab.
I20260808 10:08:06.650628 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46135
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.131.keytab
--principal=oryx/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:42171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:06.735898 10313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:06.736121 10313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:06.736148 10313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:06.737370 10313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:06.737421 10313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:06.737442 10313 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 10:08:06.737462 10313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:08:06.739114 10313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:42171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.131.keytab
--principal=oryx/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.30.190:46135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:06.739423 10313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:06.739689 10313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:06.744935 10319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:06.744956 10320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:06.744951 10313 server_base.cc:1034] running on GCE node
W20260808 10:08:06.744935 10322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:06.745512 10313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:06.745775 10313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:06.746922 10313 hybrid_clock.cc:648] HybridClock initialized: now 1786183686746886 us; error 54 us; skew 500 ppm
Aug 08 10:08:06 dist-test-slave-w206 krb5kdc[9901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183686, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:06.749675 10313 init.cc:377] Logged in from keytab as oryx/127.2.30.131@KRBTEST.COM (short username oryx)
I20260808 10:08:06.750192 10313 webserver.cc:460] Webserver started at http://127.2.30.131:41405/ using document root <none> and password file <none>
I20260808 10:08:06.750461 10313 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:06.750533 10313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:06.750689 10313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:06.752228 10313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "7c4b255262bf4f5597302c241f2db53e"
format_stamp: "Formatted at 2026-08-08 10:08:06 on dist-test-slave-w206"
server_key: "19d224781eb28eb8071f5820d588fcb8"
server_key_iv: "f291d399b70c90fcf7ac045997360fb0"
server_key_version: "encryptionkey@0"
I20260808 10:08:06.752699 10313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "7c4b255262bf4f5597302c241f2db53e"
format_stamp: "Formatted at 2026-08-08 10:08:06 on dist-test-slave-w206"
server_key: "19d224781eb28eb8071f5820d588fcb8"
server_key_iv: "f291d399b70c90fcf7ac045997360fb0"
server_key_version: "encryptionkey@0"
I20260808 10:08:06.754464 10313 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260808 10:08:06.755558 10329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:06.755812 10313 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:08:06.755947 10313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "7c4b255262bf4f5597302c241f2db53e"
format_stamp: "Formatted at 2026-08-08 10:08:06 on dist-test-slave-w206"
server_key: "19d224781eb28eb8071f5820d588fcb8"
server_key_iv: "f291d399b70c90fcf7ac045997360fb0"
server_key_version: "encryptionkey@0"
I20260808 10:08:06.756043 10313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:06.768316 10313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:06.771361 10313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:06.771513 10313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:06.772138 10313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:06.772213 10313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:06.772267 10313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:06.772292 10313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:06.779711 10313 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:36139
I20260808 10:08:06.779780 10446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:36139 every 8 connection(s)
I20260808 10:08:06.780151 10313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
Aug 08 10:08:06 dist-test-slave-w206 krb5kdc[9901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183686, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM
I20260808 10:08:06.783591 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.772948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:33269 (local address 127.2.30.190:46135)
0808 10:08:06.773028 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:06.773030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:06.773077 (+ 47us) server_negotiation.cc:407] Connection header received
0808 10:08:06.773687 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:06.773690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:06.773708 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:06.773806 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:06.774730 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.775746 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.777410 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.777759 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.777857 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.780920 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:06.780922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:06.780922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:06.780929 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:06.782202 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:06.782630 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:06.782631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:06.782631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:06.782682 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:06.782876 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:06.782877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:06.782877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:06.783054 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:06.783119 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:06.783452 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 10:08:06.783523 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:08:06.783941 10338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.772645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46135 (local address 127.2.30.131:33269)
0808 10:08:06.772942 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:06.773001 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:06.773533 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:06.773904 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:06.773909 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:06.774293 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:06.774609 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.774617 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.775877 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.775878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.777293 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.777298 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.777859 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.777860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.777974 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.778713 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:06.778724 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:06.780801 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:06.782288 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:06.782297 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:06.782299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:06.782562 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:06.782725 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:06.782725 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:06.782725 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:06.782823 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:06.783123 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:06.783130 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:06.783286 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 10:08:06.783427 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 10:08:06.783591 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":79,"threads_started":1}
I20260808 10:08:06.784816 10448 heartbeater.cc:347] Connected to a master server at 127.2.30.190:46135
I20260808 10:08:06.784915 10448 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:06.785141 10448 heartbeater.cc:511] Master 127.2.30.190:46135 requested a full tablet report, sending...
I20260808 10:08:06.785648 9948 ts_manager.cc:194] Registered new tserver with Master: 7c4b255262bf4f5597302c241f2db53e (127.2.30.131:36139)
I20260808 10:08:06.786226 9948 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:33269
I20260808 10:08:06.787225 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 10313
I20260808 10:08:06.787319 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:08:06.787523 2170 external_mini_cluster.cc:1428] Setting key 33f80e523498a4922d35720affa2d692
I20260808 10:08:06.788535 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:08:06.793463 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.786202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:53461 (local address 127.2.30.190:46135)
0808 10:08:06.786249 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:06.786251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:06.786346 (+ 95us) server_negotiation.cc:407] Connection header received
0808 10:08:06.786483 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:06.786484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:06.786496 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:06.786553 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:06.787015 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.787702 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.788754 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.788952 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.789046 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.790495 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:06.790497 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:06.790497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:06.790504 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:06.792015 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:06.792492 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:06.792494 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:06.792494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:06.792561 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:06.792846 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:06.792847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:06.792847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:06.792983 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:06.793058 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:06.793307 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 10:08:06.793391 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:08:06.793547 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.786104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46135 (local address 127.2.30.131:53461)
0808 10:08:06.786267 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:06.786280 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:06.786378 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:06.786556 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:06.786557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:06.786805 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:06.786873 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.786875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.787849 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.787851 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.788601 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.788605 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.789135 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.789136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.789223 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.789919 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:06.789926 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:06.790393 (+ 467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:06.792107 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:06.792108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:06.792108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:06.792391 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:06.792645 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:06.792645 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:06.792646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:06.792728 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:06.793073 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:06.793074 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:06.793134 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 10:08:06.793220 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:08:06.793359 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
Aug 08 10:08:06 dist-test-slave-w206 krb5kdc[9901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183686, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 10:08:06 dist-test-slave-w206 krb5kdc[9901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:06 dist-test-slave-w206 krb5kdc[9901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:08:06.815703 10000 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:06.808978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44218 (local address 127.2.30.190:46135)
0808 10:08:06.809067 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:06.809069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:06.809078 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:08:06.809105 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:06.809105 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:06.809115 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:06.809348 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:06.810027 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.810590 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.811514 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.811771 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.811860 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.815597 (+ 3737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44218: BlockingRecv error: recv got EOF from 127.0.0.1:44218 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 10:08:06 dist-test-slave-w206 krb5kdc[9901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183686, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM
I20260808 10:08:06.824815 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.816727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44226 (local address 127.2.30.190:46135)
0808 10:08:06.816790 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:06.816791 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:06.816885 (+ 94us) server_negotiation.cc:407] Connection header received
0808 10:08:06.816937 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:06.816938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:06.816947 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:06.817002 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:06.817513 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.818018 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.818844 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.819281 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.819340 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.821942 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:06.821943 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:06.821943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:06.821950 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:06.823553 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:06.823927 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:06.823928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:06.823928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:06.823981 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:06.824239 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:06.824240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:06.824240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:06.824352 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:06.824494 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:06.824678 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 10:08:06.824745 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:08:06.826751 9948 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:08:06.828512 9948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:08:06.836089 10473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.831845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:41569 (local address 127.0.0.1:35094)
0808 10:08:06.832195 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:06.832206 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:06.832268 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:06.832548 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:06.832549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:06.832551 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:06.832602 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.832604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.833919 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.833920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.834944 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.834947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.835754 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.835755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.835836 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.835851 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 10:08:06.835921 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 10:08:06.835975 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":71,"threads_started":1}
I20260808 10:08:06.836236 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.831546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38198 (local address 127.2.30.129:45649)
0808 10:08:06.831815 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:06.831821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:06.831834 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:08:06.831883 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:06.831885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:06.831904 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:06.831996 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:06.832279 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.833462 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.834999 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.835843 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.835930 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.835999 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:06.836070 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 10:08:06.836106 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":109,"threads_started":1}
I20260808 10:08:06.836205 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.831381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:45649 (local address 127.0.0.1:38198)
0808 10:08:06.831536 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:06.831569 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:06.831631 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:06.832089 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:06.832093 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:06.832096 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:06.832173 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.832176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.833556 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.833557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.834869 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.834872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.835932 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.835933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.835987 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.835990 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:08:06.836054 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 10:08:06.836091 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":98,"threads_started":1}
I20260808 10:08:06.836447 10475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.832001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35094 (local address 127.2.30.130:41569)
0808 10:08:06.832326 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:06.832330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:06.832341 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:08:06.832381 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:06.832383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:06.832398 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:06.832471 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:06.832703 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.833840 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.835081 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.835643 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.835751 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.835835 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:06.836302 (+ 467us) server_negotiation.cc:299] Negotiation successful
0808 10:08:06.836342 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":91,"threads_started":1}
I20260808 10:08:06.837134 10234 tablet_service.cc:1467] Processing CreateTablet for tablet 59de63da4177415ebf08945c684bebcb (DEFAULT_TABLE table=test-table [id=461d29e519a04c68acefadb288b9e65b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:06.837136 10091 tablet_service.cc:1467] Processing CreateTablet for tablet 59de63da4177415ebf08945c684bebcb (DEFAULT_TABLE table=test-table [id=461d29e519a04c68acefadb288b9e65b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:06.837508 10091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59de63da4177415ebf08945c684bebcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:06.837508 10234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59de63da4177415ebf08945c684bebcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:06.837777 10474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.831657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:36139 (local address 127.0.0.1:39100)
0808 10:08:06.831987 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:06.831998 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:06.832052 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:06.833813 (+ 1761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:06.833814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:06.833816 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:06.833852 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.833853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.835223 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.835224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.836664 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.836667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.837522 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.837523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.837613 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.837617 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:08:06.837671 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 10:08:06.837715 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":78,"threads_started":1}
I20260808 10:08:06.838073 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.831996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39100 (local address 127.2.30.131:36139)
0808 10:08:06.833633 (+ 1637us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:06.833637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:06.833648 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:08:06.833686 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:06.833688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:06.833704 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:06.833791 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:06.833953 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.835137 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.836775 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.837333 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.837407 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.837489 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:06.837914 (+ 425us) server_negotiation.cc:299] Negotiation successful
0808 10:08:06.837959 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1598,"thread_start_us":99,"threads_started":1}
I20260808 10:08:06.841714 10478 tablet_bootstrap.cc:492] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b: Bootstrap starting.
I20260808 10:08:06.841714 10477 tablet_bootstrap.cc:492] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731: Bootstrap starting.
I20260808 10:08:06.842502 10477 tablet_bootstrap.cc:654] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:06.842665 10478 tablet_bootstrap.cc:654] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:06.842537 10379 tablet_service.cc:1467] Processing CreateTablet for tablet 59de63da4177415ebf08945c684bebcb (DEFAULT_TABLE table=test-table [id=461d29e519a04c68acefadb288b9e65b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:06.842926 10379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59de63da4177415ebf08945c684bebcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:06.842937 10477 log.cc:824] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:06.843045 10478 log.cc:824] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:06.843811 10477 tablet_bootstrap.cc:492] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731: No bootstrap required, opened a new log
I20260808 10:08:06.843879 10477 ts_tablet_manager.cc:1397] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:08:06.845434 10477 raft_consensus.cc:348] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } }
I20260808 10:08:06.845687 10477 raft_consensus.cc:374] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:06.845742 10477 raft_consensus.cc:729] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efbe6fa7b3704e8a84a2638603210731, State: Initialized, Role: FOLLOWER
I20260808 10:08:06.845870 10477 consensus_queue.cc:260] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } }
I20260808 10:08:06.846223 10304 heartbeater.cc:503] Master 127.2.30.190:46135 was elected leader, sending a full tablet report...
I20260808 10:08:06.846194 10477 ts_tablet_manager.cc:1428] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 10:08:06.846269 10481 tablet_bootstrap.cc:492] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e: Bootstrap starting.
I20260808 10:08:06.846767 10478 tablet_bootstrap.cc:492] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b: No bootstrap required, opened a new log
I20260808 10:08:06.846868 10478 ts_tablet_manager.cc:1397] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260808 10:08:06.846908 10481 tablet_bootstrap.cc:654] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:06.847285 10481 log.cc:824] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:06.848048 10481 tablet_bootstrap.cc:492] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e: No bootstrap required, opened a new log
I20260808 10:08:06.848132 10481 ts_tablet_manager.cc:1397] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:08:06.848759 10478 raft_consensus.cc:348] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } }
I20260808 10:08:06.848912 10478 raft_consensus.cc:374] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:06.848937 10478 raft_consensus.cc:729] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4c570f847b34048861611cc7eb2bc5b, State: Initialized, Role: FOLLOWER
I20260808 10:08:06.849082 10478 consensus_queue.cc:260] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } }
I20260808 10:08:06.849406 10478 ts_tablet_manager.cc:1428] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 10:08:06.849455 10158 heartbeater.cc:503] Master 127.2.30.190:46135 was elected leader, sending a full tablet report...
I20260808 10:08:06.849910 10481 raft_consensus.cc:348] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } }
I20260808 10:08:06.850067 10481 raft_consensus.cc:374] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:06.850111 10481 raft_consensus.cc:729] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c4b255262bf4f5597302c241f2db53e, State: Initialized, Role: FOLLOWER
I20260808 10:08:06.850238 10481 consensus_queue.cc:260] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } }
I20260808 10:08:06.850549 10448 heartbeater.cc:503] Master 127.2.30.190:46135 was elected leader, sending a full tablet report...
I20260808 10:08:06.850539 10481 ts_tablet_manager.cc:1428] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260808 10:08:06.862720 10305 tablet.cc:2369] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:08:06.950515 10159 tablet.cc:2369] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:08:06.972092 10485 raft_consensus.cc:482] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:08:06.972234 10485 raft_consensus.cc:504] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } }
I20260808 10:08:06.972641 10485 leader_election.cc:288] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4c570f847b34048861611cc7eb2bc5b (127.2.30.129:45649), efbe6fa7b3704e8a84a2638603210731 (127.2.30.130:41569)
I20260808 10:08:06.976053 10338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.972921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:45649 (local address 127.2.30.131:56993)
0808 10:08:06.972995 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:06.973008 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:06.973069 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:06.973320 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:06.973321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:06.973323 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:06.973430 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.973432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.974177 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.974237 (+ 60us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.975090 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.975093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.975687 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.975688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.975834 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.975838 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:08:06.975910 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 10:08:06.975964 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 10:08:06.976145 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.973019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:56993 (local address 127.2.30.129:45649)
0808 10:08:06.973069 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:06.973071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:06.973085 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:08:06.973152 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:06.973153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:06.973181 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:06.973272 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:06.973540 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.974085 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.975178 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.975576 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.975634 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.975642 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:06.975942 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 10:08:06.976022 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:08:06.976308 10475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.973239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:47123 (local address 127.2.30.130:41569)
0808 10:08:06.973312 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:06.973314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:06.973418 (+ 104us) server_negotiation.cc:407] Connection header received
0808 10:08:06.973449 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:06.973450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:06.973463 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:06.973525 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:06.973723 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.974266 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.975377 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.975839 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.975920 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.975928 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:06.976156 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 10:08:06.976212 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:08:06.976507 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:06.973065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:41569 (local address 127.2.30.131:47123)
0808 10:08:06.973329 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:06.973340 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:06.973401 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:06.973539 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:06.973540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:06.973542 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:06.973608 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.973610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.974353 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.974354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.975175 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:06.975177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:06.975979 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:06.975980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:06.976062 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:06.976066 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:08:06.976380 (+ 314us) client_negotiation.cc:240] Negotiation successful
0808 10:08:06.976422 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":135,"threads_started":1}
I20260808 10:08:06.976941 10255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59de63da4177415ebf08945c684bebcb" candidate_uuid: "7c4b255262bf4f5597302c241f2db53e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efbe6fa7b3704e8a84a2638603210731" is_pre_election: true
I20260808 10:08:06.976935 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59de63da4177415ebf08945c684bebcb" candidate_uuid: "7c4b255262bf4f5597302c241f2db53e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4c570f847b34048861611cc7eb2bc5b" is_pre_election: true
I20260808 10:08:06.977115 10112 raft_consensus.cc:2454] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c4b255262bf4f5597302c241f2db53e in term 0.
I20260808 10:08:06.977115 10255 raft_consensus.cc:2454] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c4b255262bf4f5597302c241f2db53e in term 0.
I20260808 10:08:06.977416 10331 leader_election.cc:302] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c4b255262bf4f5597302c241f2db53e, b4c570f847b34048861611cc7eb2bc5b; no voters:
I20260808 10:08:06.977593 10485 raft_consensus.cc:2781] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:08:06.977669 10485 raft_consensus.cc:482] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:08:06.977687 10485 raft_consensus.cc:3037] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:06.978751 10485 raft_consensus.cc:504] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } }
I20260808 10:08:06.978900 10485 leader_election.cc:288] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [CANDIDATE]: Term 1 election: Requested vote from peers b4c570f847b34048861611cc7eb2bc5b (127.2.30.129:45649), efbe6fa7b3704e8a84a2638603210731 (127.2.30.130:41569)
I20260808 10:08:06.979089 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59de63da4177415ebf08945c684bebcb" candidate_uuid: "7c4b255262bf4f5597302c241f2db53e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4c570f847b34048861611cc7eb2bc5b"
I20260808 10:08:06.979135 10255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59de63da4177415ebf08945c684bebcb" candidate_uuid: "7c4b255262bf4f5597302c241f2db53e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efbe6fa7b3704e8a84a2638603210731"
I20260808 10:08:06.979180 10112 raft_consensus.cc:3037] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:06.979249 10255 raft_consensus.cc:3037] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:06.979939 10112 raft_consensus.cc:2454] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c4b255262bf4f5597302c241f2db53e in term 1.
I20260808 10:08:06.980209 10255 raft_consensus.cc:2454] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c4b255262bf4f5597302c241f2db53e in term 1.
I20260808 10:08:06.980444 10331 leader_election.cc:302] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c4b255262bf4f5597302c241f2db53e, efbe6fa7b3704e8a84a2638603210731; no voters:
I20260808 10:08:06.980635 10485 raft_consensus.cc:2781] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:08:06.980819 10485 raft_consensus.cc:686] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [term 1 LEADER]: Becoming Leader. State: Replica: 7c4b255262bf4f5597302c241f2db53e, State: Running, Role: LEADER
I20260808 10:08:06.980937 10485 consensus_queue.cc:237] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } }
I20260808 10:08:06.981894 9948 catalog_manager.cc:5495] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c4b255262bf4f5597302c241f2db53e (127.2.30.131). New cstate: current_term: 1 leader_uuid: "7c4b255262bf4f5597302c241f2db53e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } health_report { overall_health: UNKNOWN } } }
I20260808 10:08:07.021788 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:07.019277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39102 (local address 127.2.30.131:36139)
0808 10:08:07.019345 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:07.019347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:07.019354 (+ 7us) server_negotiation.cc:407] Connection header received
0808 10:08:07.019390 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:07.019391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:07.019411 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:07.019486 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:08:07.019632 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.020139 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.020988 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.021197 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.021274 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.021456 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:08:07.021561 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:08:07.021653 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:07.021672 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 10:08:07.021694 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:08:07.023939 10255 raft_consensus.cc:1261] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Refusing update from remote peer 7c4b255262bf4f5597302c241f2db53e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:08:07.023994 10112 raft_consensus.cc:1261] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Refusing update from remote peer 7c4b255262bf4f5597302c241f2db53e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:08:07.024371 10487 consensus_queue.cc:1035] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [LEADER]: Connected to new peer: Peer: permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:08:07.024514 10485 consensus_queue.cc:1035] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:08:07.027029 10490 mvcc.cc:205] Tried to move back new op lower bound from 7316208382046621696 to 7316208381874483200. Current Snapshot: MvccSnapshot[applied={T|T < 7316208382046621696}]
I20260808 10:08:07.027379 10493 mvcc.cc:205] Tried to move back new op lower bound from 7316208382046621696 to 7316208381874483200. Current Snapshot: MvccSnapshot[applied={T|T < 7316208382046621696}]
I20260808 10:08:07.027493 10491 mvcc.cc:205] Tried to move back new op lower bound from 7316208382046621696 to 7316208381874483200. Current Snapshot: MvccSnapshot[applied={T|T < 7316208382046621696 or (T in {7316208382046621696})}]
I20260808 10:08:07.029664 9948 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:08:07.029789 9948 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:08:07.030592 9948 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94: Sending DeleteTablet for 3 replicas of tablet 59de63da4177415ebf08945c684bebcb
I20260808 10:08:07.031031 10091 tablet_service.cc:1514] Processing DeleteTablet for tablet 59de63da4177415ebf08945c684bebcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:07 UTC) from {username='oryx', principal='oryx/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:38198
I20260808 10:08:07.031123 10379 tablet_service.cc:1514] Processing DeleteTablet for tablet 59de63da4177415ebf08945c684bebcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:07 UTC) from {username='oryx', principal='oryx/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:39100
I20260808 10:08:07.031252 10501 tablet_replica.cc:330] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b: stopping tablet replica
I20260808 10:08:07.031369 10501 raft_consensus.cc:2229] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:08:07.031494 10501 raft_consensus.cc:2258] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:07.031517 10234 tablet_service.cc:1514] Processing DeleteTablet for tablet 59de63da4177415ebf08945c684bebcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:07 UTC) from {username='oryx', principal='oryx/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:35094
I20260808 10:08:07.031741 9948 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:08:07.031910 9948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:08:07.031982 10501 ts_tablet_manager.cc:1905] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:07.032218 10502 tablet_replica.cc:330] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e: stopping tablet replica
I20260808 10:08:07.032320 10502 raft_consensus.cc:2229] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:08:07.032447 10502 raft_consensus.cc:2258] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:07.032603 10503 tablet_replica.cc:330] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731: stopping tablet replica
I20260808 10:08:07.032719 10503 raft_consensus.cc:2229] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:08:07.032847 10503 raft_consensus.cc:2258] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:07.033254 10503 ts_tablet_manager.cc:1905] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:07.033684 10091 tablet_service.cc:1467] Processing CreateTablet for tablet a386e5538f3f42d6813a077956d92159 (DEFAULT_TABLE table=test-table [id=1febbad0f766416f8792101ac72444c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:07.033814 10379 tablet_service.cc:1467] Processing CreateTablet for tablet a386e5538f3f42d6813a077956d92159 (DEFAULT_TABLE table=test-table [id=1febbad0f766416f8792101ac72444c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:07.033814 10234 tablet_service.cc:1467] Processing CreateTablet for tablet a386e5538f3f42d6813a077956d92159 (DEFAULT_TABLE table=test-table [id=1febbad0f766416f8792101ac72444c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:07.033933 10234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a386e5538f3f42d6813a077956d92159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:07.033933 10379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a386e5538f3f42d6813a077956d92159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:07.035298 10481 tablet_bootstrap.cc:492] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e: Bootstrap starting.
I20260808 10:08:07.035768 10481 tablet_bootstrap.cc:654] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:07.035789 10091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a386e5538f3f42d6813a077956d92159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:07.036103 10477 tablet_bootstrap.cc:492] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731: Bootstrap starting.
I20260808 10:08:07.036527 10477 tablet_bootstrap.cc:654] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:07.037045 10481 tablet_bootstrap.cc:492] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e: No bootstrap required, opened a new log
I20260808 10:08:07.037093 10481 ts_tablet_manager.cc:1397] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:08:07.037266 10478 tablet_bootstrap.cc:492] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b: Bootstrap starting.
I20260808 10:08:07.037246 10481 raft_consensus.cc:348] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.037315 10481 raft_consensus.cc:374] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:07.037338 10481 raft_consensus.cc:729] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c4b255262bf4f5597302c241f2db53e, State: Initialized, Role: FOLLOWER
I20260808 10:08:07.037382 10481 consensus_queue.cc:260] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.037470 10481 ts_tablet_manager.cc:1428] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:07.037545 10503 ts_tablet_manager.cc:1918] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:08:07.037618 10503 log.cc:1196] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/59de63da4177415ebf08945c684bebcb
I20260808 10:08:07.037647 10478 tablet_bootstrap.cc:654] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:07.037724 10477 tablet_bootstrap.cc:492] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731: No bootstrap required, opened a new log
I20260808 10:08:07.037767 10477 ts_tablet_manager.cc:1397] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:08:07.037936 10503 ts_tablet_manager.cc:1939] T 59de63da4177415ebf08945c684bebcb P efbe6fa7b3704e8a84a2638603210731: Deleting consensus metadata
I20260808 10:08:07.037910 10477 raft_consensus.cc:348] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.037989 10477 raft_consensus.cc:374] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:07.038012 10477 raft_consensus.cc:729] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efbe6fa7b3704e8a84a2638603210731, State: Initialized, Role: FOLLOWER
I20260808 10:08:07.038066 10477 consensus_queue.cc:260] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.038166 10477 ts_tablet_manager.cc:1428] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:07.038376 9932 catalog_manager.cc:4841] TS efbe6fa7b3704e8a84a2638603210731 (127.2.30.130:41569): tablet 59de63da4177415ebf08945c684bebcb (table test-table [id=461d29e519a04c68acefadb288b9e65b]) successfully deleted
I20260808 10:08:07.039044 10478 tablet_bootstrap.cc:492] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b: No bootstrap required, opened a new log
I20260808 10:08:07.039100 10478 ts_tablet_manager.cc:1397] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 10:08:07.039248 10478 raft_consensus.cc:348] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.039331 10478 raft_consensus.cc:374] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:07.039352 10478 raft_consensus.cc:729] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4c570f847b34048861611cc7eb2bc5b, State: Initialized, Role: FOLLOWER
I20260808 10:08:07.039408 10478 consensus_queue.cc:260] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.039543 10478 ts_tablet_manager.cc:1428] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:08:07.040253 10502 ts_tablet_manager.cc:1905] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:07.041314 10501 ts_tablet_manager.cc:1918] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:08:07.041381 10501 log.cc:1196] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/wal/wals/59de63da4177415ebf08945c684bebcb
I20260808 10:08:07.041824 10501 ts_tablet_manager.cc:1939] T 59de63da4177415ebf08945c684bebcb P b4c570f847b34048861611cc7eb2bc5b: Deleting consensus metadata
I20260808 10:08:07.042232 9932 catalog_manager.cc:4841] TS b4c570f847b34048861611cc7eb2bc5b (127.2.30.129:45649): tablet 59de63da4177415ebf08945c684bebcb (table test-table [id=461d29e519a04c68acefadb288b9e65b]) successfully deleted
I20260808 10:08:07.043295 10502 ts_tablet_manager.cc:1918] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:08:07.043393 10502 log.cc:1196] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/59de63da4177415ebf08945c684bebcb
I20260808 10:08:07.043771 10502 ts_tablet_manager.cc:1939] T 59de63da4177415ebf08945c684bebcb P 7c4b255262bf4f5597302c241f2db53e: Deleting consensus metadata
I20260808 10:08:07.044202 9932 catalog_manager.cc:4841] TS 7c4b255262bf4f5597302c241f2db53e (127.2.30.131:36139): tablet 59de63da4177415ebf08945c684bebcb (table test-table [id=461d29e519a04c68acefadb288b9e65b]) successfully deleted
W20260808 10:08:07.281065 10449 tablet.cc:2369] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:08:07.301087 10482 raft_consensus.cc:482] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:08:07.301251 10482 raft_consensus.cc:504] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.301702 10482 leader_election.cc:288] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4c570f847b34048861611cc7eb2bc5b (127.2.30.129:45649), 7c4b255262bf4f5597302c241f2db53e (127.2.30.131:36139)
I20260808 10:08:07.304953 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:07.302255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:59959 (local address 127.2.30.131:36139)
0808 10:08:07.302311 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:07.302313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:07.302323 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:08:07.302356 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:07.302357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:07.302365 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:07.302444 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:07.302621 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.303198 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.304203 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.304572 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.304661 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.304672 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:07.304836 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 10:08:07.304881 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 10:08:07.304971 10508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:07.302106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:36139 (local address 127.2.30.130:59959)
0808 10:08:07.302209 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:07.302220 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:07.302282 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:07.302452 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:07.302453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:07.302456 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:07.302545 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:07.302547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.303288 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.303289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:07.304101 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:07.304104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.304668 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.304669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:07.304745 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.304749 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:08:07.304826 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 10:08:07.304874 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":1,"thread_start_us":38,"threads_started":1}
I20260808 10:08:07.305055 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:07.302098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:39213 (local address 127.2.30.129:45649)
0808 10:08:07.302161 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:07.302164 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:07.302291 (+ 127us) server_negotiation.cc:407] Connection header received
0808 10:08:07.302403 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:07.302405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:07.302415 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:07.302501 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:07.302670 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.303231 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.304227 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.304618 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.304745 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.304753 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:07.304925 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 10:08:07.304966 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:08:07.305310 10507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:07.301921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:45649 (local address 127.2.30.130:39213)
0808 10:08:07.302106 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:07.302124 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:07.302316 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:07.302506 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:07.302507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:07.302510 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:07.302585 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:07.302591 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.303328 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.303329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:07.304118 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:07.304119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.304745 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.304746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:07.304831 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.304834 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:08:07.304915 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 10:08:07.304962 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":72,"threads_started":1}
I20260808 10:08:07.305233 10400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a386e5538f3f42d6813a077956d92159" candidate_uuid: "efbe6fa7b3704e8a84a2638603210731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c4b255262bf4f5597302c241f2db53e" is_pre_election: true
I20260808 10:08:07.305397 10400 raft_consensus.cc:2454] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efbe6fa7b3704e8a84a2638603210731 in term 0.
I20260808 10:08:07.305591 10186 leader_election.cc:302] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c4b255262bf4f5597302c241f2db53e, efbe6fa7b3704e8a84a2638603210731; no voters:
I20260808 10:08:07.305606 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a386e5538f3f42d6813a077956d92159" candidate_uuid: "efbe6fa7b3704e8a84a2638603210731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4c570f847b34048861611cc7eb2bc5b" is_pre_election: true
I20260808 10:08:07.305709 10112 raft_consensus.cc:2454] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efbe6fa7b3704e8a84a2638603210731 in term 0.
I20260808 10:08:07.305735 10482 raft_consensus.cc:2781] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:08:07.305773 10482 raft_consensus.cc:482] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:08:07.305804 10482 raft_consensus.cc:3037] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:07.306550 10482 raft_consensus.cc:504] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.306711 10482 leader_election.cc:288] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [CANDIDATE]: Term 1 election: Requested vote from peers b4c570f847b34048861611cc7eb2bc5b (127.2.30.129:45649), 7c4b255262bf4f5597302c241f2db53e (127.2.30.131:36139)
I20260808 10:08:07.306883 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a386e5538f3f42d6813a077956d92159" candidate_uuid: "efbe6fa7b3704e8a84a2638603210731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4c570f847b34048861611cc7eb2bc5b"
I20260808 10:08:07.306910 10400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a386e5538f3f42d6813a077956d92159" candidate_uuid: "efbe6fa7b3704e8a84a2638603210731" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c4b255262bf4f5597302c241f2db53e"
I20260808 10:08:07.306967 10112 raft_consensus.cc:3037] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:07.306968 10400 raft_consensus.cc:3037] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:07.307693 10400 raft_consensus.cc:2454] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efbe6fa7b3704e8a84a2638603210731 in term 1.
I20260808 10:08:07.307809 10112 raft_consensus.cc:2454] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efbe6fa7b3704e8a84a2638603210731 in term 1.
I20260808 10:08:07.307864 10186 leader_election.cc:302] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c4b255262bf4f5597302c241f2db53e, efbe6fa7b3704e8a84a2638603210731; no voters:
I20260808 10:08:07.308025 10482 raft_consensus.cc:2781] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:08:07.308161 10482 raft_consensus.cc:686] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [term 1 LEADER]: Becoming Leader. State: Replica: efbe6fa7b3704e8a84a2638603210731, State: Running, Role: LEADER
I20260808 10:08:07.308241 10482 consensus_queue.cc:237] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.309144 9947 catalog_manager.cc:5495] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 reported cstate change: term changed from 0 to 1, leader changed from <none> to efbe6fa7b3704e8a84a2638603210731 (127.2.30.130). New cstate: current_term: 1 leader_uuid: "efbe6fa7b3704e8a84a2638603210731" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } health_report { overall_health: UNKNOWN } } }
Aug 08 10:08:07 dist-test-slave-w206 krb5kdc[9901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183686, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.190@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM
I20260808 10:08:07.343811 10517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:07.335911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44232 (local address 127.2.30.190:46135)
0808 10:08:07.336098 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:07.336101 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:07.336194 (+ 93us) server_negotiation.cc:407] Connection header received
0808 10:08:07.336300 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:07.336301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:07.336313 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:07.336411 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:07.336942 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.337503 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.338248 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.338453 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.338541 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.340943 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:07.340945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:07.340945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:07.340953 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:07.342412 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:07.342871 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:07.342872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:07.342872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:07.342932 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:07.343184 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:07.343185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:07.343185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:07.343308 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:07.343389 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:07.343547 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 10:08:07.343679 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":99,"threads_started":1}
I20260808 10:08:07.343815 10516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:07.335810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46135 (local address 127.0.0.1:44232)
0808 10:08:07.336094 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:07.336109 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:07.336217 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:07.336415 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:07.336416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:07.336797 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:07.336848 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:07.336851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.337595 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.337596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:07.338142 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:07.338144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.338543 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.338544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:07.338596 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.339325 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:07.339330 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:07.340840 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:07.342521 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:07.342530 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:07.342532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:07.342752 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:07.343048 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:07.343049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:07.343050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:07.343117 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:07.343378 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:07.343386 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:07.343436 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 10:08:07.343522 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:08:07.343732 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":97,"thread_start_us":123,"threads_started":1}
I20260808 10:08:07.346519 9947 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:44232:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 10:08:07.346817 9947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:08:07.348850 10379 tablet_service.cc:1467] Processing CreateTablet for tablet c5af3f2084a8421abab39e94b5322b39 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6eb4b56ed28c4d24b804c657a7d3d13d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:08:07.348821 10234 tablet_service.cc:1467] Processing CreateTablet for tablet c5af3f2084a8421abab39e94b5322b39 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6eb4b56ed28c4d24b804c657a7d3d13d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:08:07.348867 10091 tablet_service.cc:1467] Processing CreateTablet for tablet c5af3f2084a8421abab39e94b5322b39 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6eb4b56ed28c4d24b804c657a7d3d13d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 10:08:07.349014 10379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5af3f2084a8421abab39e94b5322b39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:07.349014 10234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5af3f2084a8421abab39e94b5322b39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:07.349041 10091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5af3f2084a8421abab39e94b5322b39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:07.351387 10481 tablet_bootstrap.cc:492] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e: Bootstrap starting.
I20260808 10:08:07.351907 10481 tablet_bootstrap.cc:654] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:07.352294 10477 tablet_bootstrap.cc:492] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731: Bootstrap starting.
I20260808 10:08:07.352800 10481 tablet_bootstrap.cc:492] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e: No bootstrap required, opened a new log
I20260808 10:08:07.352864 10481 ts_tablet_manager.cc:1397] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:08:07.352870 10477 tablet_bootstrap.cc:654] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:07.352998 10481 raft_consensus.cc:348] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.353097 10481 raft_consensus.cc:374] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:07.353130 10481 raft_consensus.cc:729] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c4b255262bf4f5597302c241f2db53e, State: Initialized, Role: FOLLOWER
I20260808 10:08:07.353191 10481 consensus_queue.cc:260] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.353305 10481 ts_tablet_manager.cc:1428] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:07.353435 10498 tablet_replica.cc:439] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } }
I20260808 10:08:07.353794 10478 tablet_bootstrap.cc:492] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b: Bootstrap starting.
I20260808 10:08:07.353564 10498 tablet_replica.cc:442] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:08:07.354266 10477 tablet_bootstrap.cc:492] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731: No bootstrap required, opened a new log
I20260808 10:08:07.354301 10478 tablet_bootstrap.cc:654] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:07.354336 10477 ts_tablet_manager.cc:1397] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:08:07.354549 10477 raft_consensus.cc:348] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.354651 10477 raft_consensus.cc:374] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:07.354686 10477 raft_consensus.cc:729] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efbe6fa7b3704e8a84a2638603210731, State: Initialized, Role: FOLLOWER
I20260808 10:08:07.354748 10477 consensus_queue.cc:260] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.354862 10477 ts_tablet_manager.cc:1428] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:08:07.355047 10482 tablet_replica.cc:439] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } }
I20260808 10:08:07.355123 10482 tablet_replica.cc:442] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:08:07.355327 10478 tablet_bootstrap.cc:492] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b: No bootstrap required, opened a new log
I20260808 10:08:07.355381 10478 ts_tablet_manager.cc:1397] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:08:07.355573 10478 raft_consensus.cc:348] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.355691 10478 raft_consensus.cc:374] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:07.355715 10478 raft_consensus.cc:729] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4c570f847b34048861611cc7eb2bc5b, State: Initialized, Role: FOLLOWER
I20260808 10:08:07.355785 10478 consensus_queue.cc:260] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.355892 10478 ts_tablet_manager.cc:1428] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:07.355988 10484 tablet_replica.cc:439] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } }
I20260808 10:08:07.356091 10484 tablet_replica.cc:442] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:08:07.517882 10498 raft_consensus.cc:482] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:08:07.517997 10498 raft_consensus.cc:504] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.518181 10498 leader_election.cc:288] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4c570f847b34048861611cc7eb2bc5b (127.2.30.129:45649), efbe6fa7b3704e8a84a2638603210731 (127.2.30.130:41569)
I20260808 10:08:07.518602 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5af3f2084a8421abab39e94b5322b39" candidate_uuid: "7c4b255262bf4f5597302c241f2db53e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4c570f847b34048861611cc7eb2bc5b" is_pre_election: true
I20260808 10:08:07.518743 10112 raft_consensus.cc:2454] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c4b255262bf4f5597302c241f2db53e in term 0.
I20260808 10:08:07.518694 10255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5af3f2084a8421abab39e94b5322b39" candidate_uuid: "7c4b255262bf4f5597302c241f2db53e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efbe6fa7b3704e8a84a2638603210731" is_pre_election: true
I20260808 10:08:07.518852 10255 raft_consensus.cc:2454] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c4b255262bf4f5597302c241f2db53e in term 0.
I20260808 10:08:07.518959 10331 leader_election.cc:302] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c4b255262bf4f5597302c241f2db53e, b4c570f847b34048861611cc7eb2bc5b; no voters:
I20260808 10:08:07.519133 10498 raft_consensus.cc:2781] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:08:07.519196 10498 raft_consensus.cc:482] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:08:07.519228 10498 raft_consensus.cc:3037] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:07.520089 10498 raft_consensus.cc:504] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.520303 10498 leader_election.cc:288] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [CANDIDATE]: Term 1 election: Requested vote from peers b4c570f847b34048861611cc7eb2bc5b (127.2.30.129:45649), efbe6fa7b3704e8a84a2638603210731 (127.2.30.130:41569)
I20260808 10:08:07.520504 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5af3f2084a8421abab39e94b5322b39" candidate_uuid: "7c4b255262bf4f5597302c241f2db53e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4c570f847b34048861611cc7eb2bc5b"
I20260808 10:08:07.520532 10255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5af3f2084a8421abab39e94b5322b39" candidate_uuid: "7c4b255262bf4f5597302c241f2db53e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efbe6fa7b3704e8a84a2638603210731"
I20260808 10:08:07.520654 10112 raft_consensus.cc:3037] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:07.520668 10255 raft_consensus.cc:3037] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:07.521421 10112 raft_consensus.cc:2454] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c4b255262bf4f5597302c241f2db53e in term 1.
I20260808 10:08:07.521432 10255 raft_consensus.cc:2454] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c4b255262bf4f5597302c241f2db53e in term 1.
I20260808 10:08:07.521625 10331 leader_election.cc:302] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c4b255262bf4f5597302c241f2db53e, b4c570f847b34048861611cc7eb2bc5b; no voters:
I20260808 10:08:07.521759 10498 raft_consensus.cc:2781] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:08:07.521857 10498 raft_consensus.cc:686] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [term 1 LEADER]: Becoming Leader. State: Replica: 7c4b255262bf4f5597302c241f2db53e, State: Running, Role: LEADER
I20260808 10:08:07.521924 10498 consensus_queue.cc:237] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.522255 10487 tablet_replica.cc:439] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e: TxnStatusTablet state changed. Reason: New leader 7c4b255262bf4f5597302c241f2db53e. Latest consensus state: current_term: 1 leader_uuid: "7c4b255262bf4f5597302c241f2db53e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } }
I20260808 10:08:07.522359 10487 tablet_replica.cc:442] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:08:07.522521 10522 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 10:08:07.522693 10522 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 10:08:07.522792 9948 catalog_manager.cc:5495] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c4b255262bf4f5597302c241f2db53e (127.2.30.131). New cstate: current_term: 1 leader_uuid: "7c4b255262bf4f5597302c241f2db53e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } health_report { overall_health: HEALTHY } } }
I20260808 10:08:07.538259 10516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:07.535152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:36139 (local address 127.0.0.1:39114)
0808 10:08:07.535275 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:07.535289 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:07.535349 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:07.535524 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:07.535524 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:07.535525 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:08:07.535578 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:07.535579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.536457 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.536458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:07.537285 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:07.537290 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.537784 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.537785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:07.537834 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.537852 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:08:07.538060 (+ 208us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:08:07.538072 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 10:08:07.538123 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 10:08:07.538177 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 10:08:07.538266 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:07.535294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39114 (local address 127.2.30.131:36139)
0808 10:08:07.535370 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:07.535373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:07.535385 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:08:07.535424 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:07.535425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:07.535440 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:07.535512 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:08:07.535624 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.536349 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.537385 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.537690 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.537772 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.537934 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:08:07.537972 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:08:07.538035 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:07.538141 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 10:08:07.538180 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:08:07.539522 10255 raft_consensus.cc:1261] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Refusing update from remote peer 7c4b255262bf4f5597302c241f2db53e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:08:07.539562 10112 raft_consensus.cc:1261] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Refusing update from remote peer 7c4b255262bf4f5597302c241f2db53e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:08:07.539868 10487 consensus_queue.cc:1035] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [LEADER]: Connected to new peer: Peer: permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:08:07.540041 10487 consensus_queue.cc:1035] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:08:07.540544 10484 tablet_replica.cc:439] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b: TxnStatusTablet state changed. Reason: New leader 7c4b255262bf4f5597302c241f2db53e. Latest consensus state: current_term: 1 leader_uuid: "7c4b255262bf4f5597302c241f2db53e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } }
I20260808 10:08:07.540638 10484 tablet_replica.cc:442] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:08:07.540577 10482 tablet_replica.cc:439] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731: TxnStatusTablet state changed. Reason: New leader 7c4b255262bf4f5597302c241f2db53e. Latest consensus state: current_term: 1 leader_uuid: "7c4b255262bf4f5597302c241f2db53e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } }
I20260808 10:08:07.540683 10482 tablet_replica.cc:442] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:08:07.541790 10487 tablet_replica.cc:439] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7c4b255262bf4f5597302c241f2db53e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } }
I20260808 10:08:07.541817 10498 tablet_replica.cc:439] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7c4b255262bf4f5597302c241f2db53e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } }
I20260808 10:08:07.541921 10498 tablet_replica.cc:442] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:08:07.541887 10487 tablet_replica.cc:442] T c5af3f2084a8421abab39e94b5322b39 P 7c4b255262bf4f5597302c241f2db53e: This TxnStatusTablet replica's current role is: LEADER
I20260808 10:08:07.542407 10484 tablet_replica.cc:439] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7c4b255262bf4f5597302c241f2db53e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } }
I20260808 10:08:07.542498 10484 tablet_replica.cc:442] T c5af3f2084a8421abab39e94b5322b39 P b4c570f847b34048861611cc7eb2bc5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:08:07.543325 10482 tablet_replica.cc:439] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7c4b255262bf4f5597302c241f2db53e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } } }
I20260808 10:08:07.543423 10482 tablet_replica.cc:442] T c5af3f2084a8421abab39e94b5322b39 P efbe6fa7b3704e8a84a2638603210731: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 10:08:07.546911 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:07.543993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35102 (local address 127.2.30.130:41569)
0808 10:08:07.544121 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:07.544123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:07.544130 (+ 7us) server_negotiation.cc:407] Connection header received
0808 10:08:07.544194 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:07.544195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:07.544217 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:07.544301 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:08:07.544472 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.544944 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.546041 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.546238 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.546346 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.546523 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:08:07.546627 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:08:07.546682 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:07.546760 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 10:08:07.546781 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
I20260808 10:08:07.552044 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:07.549542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:45229 (local address 127.2.30.131:36139)
0808 10:08:07.549595 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:07.549597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:07.549675 (+ 78us) server_negotiation.cc:407] Connection header received
0808 10:08:07.549746 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:07.549747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:07.549762 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:07.549816 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:08:07.549999 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.550474 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.551331 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.551525 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.551599 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.551775 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:08:07.551816 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:08:07.551881 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:07.551943 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 10:08:07.551965 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 10:08:07.552109 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:07.549464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:36139 (local address 127.2.30.130:45229)
0808 10:08:07.549612 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:07.549623 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:07.549696 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:07.549844 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:07.549844 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:07.549846 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:08:07.549905 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:07.549908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.550554 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.550555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:07.551239 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:07.551241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.551611 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.551612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:07.551678 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.551692 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:08:07.551881 (+ 189us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:08:07.551884 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:08:07.551938 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 10:08:07.551981 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260808 10:08:07.554324 10112 raft_consensus.cc:1261] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Refusing update from remote peer efbe6fa7b3704e8a84a2638603210731: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:08:07.554442 10400 raft_consensus.cc:1261] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e [term 1 FOLLOWER]: Refusing update from remote peer efbe6fa7b3704e8a84a2638603210731: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:08:07.554603 10482 consensus_queue.cc:1035] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:08:07.554734 10509 consensus_queue.cc:1035] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:08:07.563711 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:07.560899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:56017 (local address 127.2.30.130:41569)
0808 10:08:07.560937 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:07.560939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:07.561089 (+ 150us) server_negotiation.cc:407] Connection header received
0808 10:08:07.561159 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:07.561160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:07.561171 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:07.561225 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:08:07.561354 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.561826 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.562954 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.563210 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.563265 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.563481 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:08:07.563509 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:08:07.563549 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:07.563637 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 10:08:07.563656 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 10:08:07.563787 10544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:07.560837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:41569 (local address 127.2.30.131:56017)
0808 10:08:07.561010 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:07.561025 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:07.561107 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:07.561229 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:07.561230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:07.561231 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:08:07.561293 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:07.561294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.561942 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.561943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:07.562862 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:07.562864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.563295 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.563296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:07.563373 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.563391 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:08:07.563575 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:08:07.563577 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:08:07.563637 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 10:08:07.563687 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":110,"threads_started":1}
I20260808 10:08:07.572588 9947 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:08:07.572706 9947 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:08:07.573371 9947 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94: Sending DeleteTablet for 3 replicas of tablet a386e5538f3f42d6813a077956d92159
I20260808 10:08:07.573658 10091 tablet_service.cc:1514] Processing DeleteTablet for tablet a386e5538f3f42d6813a077956d92159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:07 UTC) from {username='oryx', principal='oryx/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:38198
I20260808 10:08:07.573726 10234 tablet_service.cc:1514] Processing DeleteTablet for tablet a386e5538f3f42d6813a077956d92159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:07 UTC) from {username='oryx', principal='oryx/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:35094
I20260808 10:08:07.573994 10379 tablet_service.cc:1514] Processing DeleteTablet for tablet a386e5538f3f42d6813a077956d92159 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:07 UTC) from {username='oryx', principal='oryx/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:39100
I20260808 10:08:07.574028 10548 tablet_replica.cc:330] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b: stopping tablet replica
I20260808 10:08:07.574081 10548 raft_consensus.cc:2229] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:08:07.574178 10548 raft_consensus.cc:2258] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:07.574338 10549 tablet_replica.cc:330] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731: stopping tablet replica
I20260808 10:08:07.574398 10549 raft_consensus.cc:2229] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:08:07.574477 10548 ts_tablet_manager.cc:1905] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:07.574491 10549 raft_consensus.cc:2258] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:07.574947 10549 ts_tablet_manager.cc:1905] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:07.576249 10550 tablet_replica.cc:330] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e: stopping tablet replica
I20260808 10:08:07.576311 10550 raft_consensus.cc:2229] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:08:07.576383 10550 raft_consensus.cc:2258] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:07.576646 10550 ts_tablet_manager.cc:1905] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:07.577929 10548 ts_tablet_manager.cc:1918] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:08:07.578020 10548 log.cc:1196] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/wal/wals/a386e5538f3f42d6813a077956d92159
I20260808 10:08:07.578245 10550 ts_tablet_manager.cc:1918] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:08:07.578318 10550 log.cc:1196] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/a386e5538f3f42d6813a077956d92159
I20260808 10:08:07.578336 10548 ts_tablet_manager.cc:1939] T a386e5538f3f42d6813a077956d92159 P b4c570f847b34048861611cc7eb2bc5b: Deleting consensus metadata
I20260808 10:08:07.578608 10550 ts_tablet_manager.cc:1939] T a386e5538f3f42d6813a077956d92159 P 7c4b255262bf4f5597302c241f2db53e: Deleting consensus metadata
I20260808 10:08:07.578723 9932 catalog_manager.cc:4841] TS b4c570f847b34048861611cc7eb2bc5b (127.2.30.129:45649): tablet a386e5538f3f42d6813a077956d92159 (table test-table [id=1febbad0f766416f8792101ac72444c9]) successfully deleted
I20260808 10:08:07.578918 9932 catalog_manager.cc:4841] TS 7c4b255262bf4f5597302c241f2db53e (127.2.30.131:36139): tablet a386e5538f3f42d6813a077956d92159 (table test-table [id=1febbad0f766416f8792101ac72444c9]) successfully deleted
I20260808 10:08:07.580585 10549 ts_tablet_manager.cc:1918] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:08:07.580683 10549 log.cc:1196] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/a386e5538f3f42d6813a077956d92159
I20260808 10:08:07.580935 10549 ts_tablet_manager.cc:1939] T a386e5538f3f42d6813a077956d92159 P efbe6fa7b3704e8a84a2638603210731: Deleting consensus metadata
I20260808 10:08:07.581782 9932 catalog_manager.cc:4841] TS efbe6fa7b3704e8a84a2638603210731 (127.2.30.130:41569): tablet a386e5538f3f42d6813a077956d92159 (table test-table [id=1febbad0f766416f8792101ac72444c9]) successfully deleted
Aug 08 10:08:07 dist-test-slave-w206 krb5kdc[9901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 10:08:07 dist-test-slave-w206 krb5kdc[9901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:07 dist-test-slave-w206 krb5kdc[9901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:08:07.612092 10517 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:07.604461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44248 (local address 127.2.30.190:46135)
0808 10:08:07.604520 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:07.604522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:07.604671 (+ 149us) server_negotiation.cc:407] Connection header received
0808 10:08:07.604751 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:07.604752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:07.604771 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:07.604838 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:07.605665 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.606491 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.607693 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.608013 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.608087 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.612021 (+ 3934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44248: BlockingRecv error: recv got EOF from 127.0.0.1:44248 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 08 10:08:07 dist-test-slave-w206 krb5kdc[9901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM
I20260808 10:08:07.625669 10517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:07.614096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44260 (local address 127.2.30.190:46135)
0808 10:08:07.614141 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:07.614143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:07.614359 (+ 216us) server_negotiation.cc:407] Connection header received
0808 10:08:07.614462 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:07.614463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:07.614474 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:07.614508 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:07.615398 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.616275 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.617370 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.617678 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.617771 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.622296 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:07.622298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:07.622299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:07.622308 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:07.624311 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:07.624762 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:07.624763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:07.624764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:07.624829 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:07.625048 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:07.625048 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:07.625049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:07.625199 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:07.625314 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:07.625475 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 10:08:07.625564 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:08:07.627545 9947 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:08:07.627737 9947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:08:07.629412 10091 tablet_service.cc:1467] Processing CreateTablet for tablet 9c46d53f48b44a769c9942991788d2c4 (DEFAULT_TABLE table=test-table [id=998d6b4353ca45e189c490ae512cc24f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:07.629462 10379 tablet_service.cc:1467] Processing CreateTablet for tablet 9c46d53f48b44a769c9942991788d2c4 (DEFAULT_TABLE table=test-table [id=998d6b4353ca45e189c490ae512cc24f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:07.629580 10091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c46d53f48b44a769c9942991788d2c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:07.629599 10379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c46d53f48b44a769c9942991788d2c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:07.631028 10481 tablet_bootstrap.cc:492] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e: Bootstrap starting.
I20260808 10:08:07.631304 10234 tablet_service.cc:1467] Processing CreateTablet for tablet 9c46d53f48b44a769c9942991788d2c4 (DEFAULT_TABLE table=test-table [id=998d6b4353ca45e189c490ae512cc24f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:07.631381 10481 tablet_bootstrap.cc:654] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:07.631450 10234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c46d53f48b44a769c9942991788d2c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:07.632354 10481 tablet_bootstrap.cc:492] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e: No bootstrap required, opened a new log
I20260808 10:08:07.632400 10481 ts_tablet_manager.cc:1397] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 10:08:07.632541 10481 raft_consensus.cc:348] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.632606 10481 raft_consensus.cc:374] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:07.632627 10481 raft_consensus.cc:729] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c4b255262bf4f5597302c241f2db53e, State: Initialized, Role: FOLLOWER
I20260808 10:08:07.632670 10481 consensus_queue.cc:260] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.632757 10481 ts_tablet_manager.cc:1428] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:07.632977 10478 tablet_bootstrap.cc:492] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b: Bootstrap starting.
I20260808 10:08:07.633440 10478 tablet_bootstrap.cc:654] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:07.633934 10477 tablet_bootstrap.cc:492] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731: Bootstrap starting.
I20260808 10:08:07.634399 10477 tablet_bootstrap.cc:654] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:07.635169 10477 tablet_bootstrap.cc:492] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731: No bootstrap required, opened a new log
I20260808 10:08:07.635217 10477 ts_tablet_manager.cc:1397] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:07.635422 10477 raft_consensus.cc:348] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.635502 10477 raft_consensus.cc:374] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:07.635524 10477 raft_consensus.cc:729] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efbe6fa7b3704e8a84a2638603210731, State: Initialized, Role: FOLLOWER
I20260808 10:08:07.635565 10477 consensus_queue.cc:260] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.635648 10477 ts_tablet_manager.cc:1428] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:08:07.636657 10478 tablet_bootstrap.cc:492] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b: No bootstrap required, opened a new log
I20260808 10:08:07.636724 10478 ts_tablet_manager.cc:1397] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 10:08:07.636893 10478 raft_consensus.cc:348] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.636983 10478 raft_consensus.cc:374] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:07.637007 10478 raft_consensus.cc:729] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4c570f847b34048861611cc7eb2bc5b, State: Initialized, Role: FOLLOWER
I20260808 10:08:07.637060 10478 consensus_queue.cc:260] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.637158 10478 ts_tablet_manager.cc:1428] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:07.843781 10498 raft_consensus.cc:482] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:08:07.843904 10498 raft_consensus.cc:504] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.844079 10498 leader_election.cc:288] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4c570f847b34048861611cc7eb2bc5b (127.2.30.129:45649), efbe6fa7b3704e8a84a2638603210731 (127.2.30.130:41569)
I20260808 10:08:07.844357 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c46d53f48b44a769c9942991788d2c4" candidate_uuid: "7c4b255262bf4f5597302c241f2db53e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4c570f847b34048861611cc7eb2bc5b" is_pre_election: true
I20260808 10:08:07.844398 10255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c46d53f48b44a769c9942991788d2c4" candidate_uuid: "7c4b255262bf4f5597302c241f2db53e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efbe6fa7b3704e8a84a2638603210731" is_pre_election: true
I20260808 10:08:07.844480 10112 raft_consensus.cc:2454] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c4b255262bf4f5597302c241f2db53e in term 0.
I20260808 10:08:07.844481 10255 raft_consensus.cc:2454] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c4b255262bf4f5597302c241f2db53e in term 0.
I20260808 10:08:07.844661 10331 leader_election.cc:302] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c4b255262bf4f5597302c241f2db53e, efbe6fa7b3704e8a84a2638603210731; no voters:
I20260808 10:08:07.844869 10498 raft_consensus.cc:2781] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:08:07.844921 10498 raft_consensus.cc:482] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:08:07.844954 10498 raft_consensus.cc:3037] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:07.845739 10498 raft_consensus.cc:504] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.845917 10498 leader_election.cc:288] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [CANDIDATE]: Term 1 election: Requested vote from peers b4c570f847b34048861611cc7eb2bc5b (127.2.30.129:45649), efbe6fa7b3704e8a84a2638603210731 (127.2.30.130:41569)
I20260808 10:08:07.846127 10112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c46d53f48b44a769c9942991788d2c4" candidate_uuid: "7c4b255262bf4f5597302c241f2db53e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4c570f847b34048861611cc7eb2bc5b"
I20260808 10:08:07.846159 10255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c46d53f48b44a769c9942991788d2c4" candidate_uuid: "7c4b255262bf4f5597302c241f2db53e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efbe6fa7b3704e8a84a2638603210731"
I20260808 10:08:07.846231 10112 raft_consensus.cc:3037] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:07.846240 10255 raft_consensus.cc:3037] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:07.846864 10112 raft_consensus.cc:2454] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c4b255262bf4f5597302c241f2db53e in term 1.
I20260808 10:08:07.846935 10255 raft_consensus.cc:2454] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c4b255262bf4f5597302c241f2db53e in term 1.
I20260808 10:08:07.847087 10331 leader_election.cc:302] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c4b255262bf4f5597302c241f2db53e, b4c570f847b34048861611cc7eb2bc5b; no voters:
I20260808 10:08:07.847210 10498 raft_consensus.cc:2781] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:08:07.847277 10498 raft_consensus.cc:686] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [term 1 LEADER]: Becoming Leader. State: Replica: 7c4b255262bf4f5597302c241f2db53e, State: Running, Role: LEADER
I20260808 10:08:07.847355 10498 consensus_queue.cc:237] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.847996 9947 catalog_manager.cc:5495] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c4b255262bf4f5597302c241f2db53e (127.2.30.131). New cstate: current_term: 1 leader_uuid: "7c4b255262bf4f5597302c241f2db53e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } health_report { overall_health: HEALTHY } } }
I20260808 10:08:07.862728 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:07.860388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39118 (local address 127.2.30.131:36139)
0808 10:08:07.860447 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:07.860449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:07.860461 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:08:07.860495 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:07.860496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:07.860505 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:07.860557 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:08:07.860726 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.861268 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.862003 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:07.862263 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:07.862334 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:07.862466 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:08:07.862493 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:08:07.862535 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:07.862647 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 10:08:07.862665 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 10:08:07.863672 10255 raft_consensus.cc:1261] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Refusing update from remote peer 7c4b255262bf4f5597302c241f2db53e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:08:07.863759 10112 raft_consensus.cc:1261] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Refusing update from remote peer 7c4b255262bf4f5597302c241f2db53e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:08:07.863963 10530 consensus_queue.cc:1035] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [LEADER]: Connected to new peer: Peer: permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:08:07.864084 10498 consensus_queue.cc:1035] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:08:07.867080 9947 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:08:07.867203 9947 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:08:07.867921 9947 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94: Sending DeleteTablet for 3 replicas of tablet 9c46d53f48b44a769c9942991788d2c4
I20260808 10:08:07.868258 10379 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c46d53f48b44a769c9942991788d2c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:07 UTC) from {username='oryx', principal='oryx/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:39100
I20260808 10:08:07.868367 10550 tablet_replica.cc:330] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e: stopping tablet replica
I20260808 10:08:07.868367 10234 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c46d53f48b44a769c9942991788d2c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:07 UTC) from {username='oryx', principal='oryx/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:35094
I20260808 10:08:07.868435 10550 raft_consensus.cc:2229] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:08:07.868495 10550 raft_consensus.cc:2258] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:07.868489 10549 tablet_replica.cc:330] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731: stopping tablet replica
I20260808 10:08:07.868486 9947 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 10:08:07.868543 10549 raft_consensus.cc:2229] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:08:07.868582 10091 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c46d53f48b44a769c9942991788d2c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:07 UTC) from {username='oryx', principal='oryx/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:38198
I20260808 10:08:07.868613 10549 raft_consensus.cc:2258] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 10:08:07.868621 9947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:08:07.868664 10548 tablet_replica.cc:330] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b: stopping tablet replica
I20260808 10:08:07.868706 10548 raft_consensus.cc:2229] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:08:07.868746 10548 raft_consensus.cc:2258] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:07.868973 10549 ts_tablet_manager.cc:1905] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:07.869274 10548 ts_tablet_manager.cc:1905] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:07.869282 10550 ts_tablet_manager.cc:1905] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:07.870751 10548 ts_tablet_manager.cc:1918] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:08:07.870766 10549 ts_tablet_manager.cc:1918] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:08:07.870796 10550 ts_tablet_manager.cc:1918] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:08:07.870827 10548 log.cc:1196] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-0/wal/wals/9c46d53f48b44a769c9942991788d2c4
I20260808 10:08:07.870846 10550 log.cc:1196] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/9c46d53f48b44a769c9942991788d2c4
I20260808 10:08:07.870847 10549 log.cc:1196] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/9c46d53f48b44a769c9942991788d2c4
I20260808 10:08:07.871094 10550 ts_tablet_manager.cc:1939] T 9c46d53f48b44a769c9942991788d2c4 P 7c4b255262bf4f5597302c241f2db53e: Deleting consensus metadata
I20260808 10:08:07.871094 10548 ts_tablet_manager.cc:1939] T 9c46d53f48b44a769c9942991788d2c4 P b4c570f847b34048861611cc7eb2bc5b: Deleting consensus metadata
I20260808 10:08:07.871182 10549 ts_tablet_manager.cc:1939] T 9c46d53f48b44a769c9942991788d2c4 P efbe6fa7b3704e8a84a2638603210731: Deleting consensus metadata
I20260808 10:08:07.871574 9932 catalog_manager.cc:4841] TS efbe6fa7b3704e8a84a2638603210731 (127.2.30.130:41569): tablet 9c46d53f48b44a769c9942991788d2c4 (table test-table [id=998d6b4353ca45e189c490ae512cc24f]) successfully deleted
I20260808 10:08:07.871680 9932 catalog_manager.cc:4841] TS 7c4b255262bf4f5597302c241f2db53e (127.2.30.131:36139): tablet 9c46d53f48b44a769c9942991788d2c4 (table test-table [id=998d6b4353ca45e189c490ae512cc24f]) successfully deleted
I20260808 10:08:07.871726 9932 catalog_manager.cc:4841] TS b4c570f847b34048861611cc7eb2bc5b (127.2.30.129:45649): tablet 9c46d53f48b44a769c9942991788d2c4 (table test-table [id=998d6b4353ca45e189c490ae512cc24f]) successfully deleted
I20260808 10:08:07.873077 10091 tablet_service.cc:1467] Processing CreateTablet for tablet 48d488da66b74bfcaed1664f103f855b (DEFAULT_TABLE table=test-table [id=b967b03a69cd42b087b8e57e6a7f2db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:07.873126 10234 tablet_service.cc:1467] Processing CreateTablet for tablet 48d488da66b74bfcaed1664f103f855b (DEFAULT_TABLE table=test-table [id=b967b03a69cd42b087b8e57e6a7f2db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:07.873208 10091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48d488da66b74bfcaed1664f103f855b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:07.873258 10234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48d488da66b74bfcaed1664f103f855b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:07.874087 10478 tablet_bootstrap.cc:492] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b: Bootstrap starting.
I20260808 10:08:07.874195 10477 tablet_bootstrap.cc:492] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731: Bootstrap starting.
I20260808 10:08:07.874202 10379 tablet_service.cc:1467] Processing CreateTablet for tablet 48d488da66b74bfcaed1664f103f855b (DEFAULT_TABLE table=test-table [id=b967b03a69cd42b087b8e57e6a7f2db4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:07.874317 10379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48d488da66b74bfcaed1664f103f855b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:07.874532 10477 tablet_bootstrap.cc:654] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:07.874809 10478 tablet_bootstrap.cc:654] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:07.875463 10481 tablet_bootstrap.cc:492] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e: Bootstrap starting.
I20260808 10:08:07.875527 10478 tablet_bootstrap.cc:492] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b: No bootstrap required, opened a new log
I20260808 10:08:07.875562 10478 ts_tablet_manager.cc:1397] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 10:08:07.875713 10478 raft_consensus.cc:348] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.875788 10481 tablet_bootstrap.cc:654] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:07.875793 10478 raft_consensus.cc:374] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:07.875818 10478 raft_consensus.cc:729] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4c570f847b34048861611cc7eb2bc5b, State: Initialized, Role: FOLLOWER
I20260808 10:08:07.875841 10477 tablet_bootstrap.cc:492] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731: No bootstrap required, opened a new log
I20260808 10:08:07.875874 10478 consensus_queue.cc:260] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.875905 10477 ts_tablet_manager.cc:1397] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:08:07.875988 10478 ts_tablet_manager.cc:1428] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:07.876030 10477 raft_consensus.cc:348] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.876106 10477 raft_consensus.cc:374] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:07.876143 10477 raft_consensus.cc:729] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efbe6fa7b3704e8a84a2638603210731, State: Initialized, Role: FOLLOWER
I20260808 10:08:07.876219 10477 consensus_queue.cc:260] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.876338 10477 ts_tablet_manager.cc:1428] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 10:08:07.876394 10481 tablet_bootstrap.cc:492] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e: No bootstrap required, opened a new log
I20260808 10:08:07.876436 10481 ts_tablet_manager.cc:1397] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 10:08:07.876540 10481 raft_consensus.cc:348] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.876605 10481 raft_consensus.cc:374] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:07.876624 10481 raft_consensus.cc:729] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c4b255262bf4f5597302c241f2db53e, State: Initialized, Role: FOLLOWER
I20260808 10:08:07.876664 10481 consensus_queue.cc:260] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:07.876770 10481 ts_tablet_manager.cc:1428] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:08.132052 10484 raft_consensus.cc:482] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:08:08.132210 10484 raft_consensus.cc:504] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:08.132604 10484 leader_election.cc:288] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efbe6fa7b3704e8a84a2638603210731 (127.2.30.130:41569), 7c4b255262bf4f5597302c241f2db53e (127.2.30.131:36139)
I20260808 10:08:08.137223 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:08.133215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:49355 (local address 127.2.30.131:36139)
0808 10:08:08.133284 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:08.133286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:08.133381 (+ 95us) server_negotiation.cc:407] Connection header received
0808 10:08:08.133434 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:08.133435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:08.133449 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:08.133488 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:08.133662 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.134365 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.136171 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.136759 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.136848 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:08.136860 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:08.137103 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 10:08:08.137150 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 10:08:08.137240 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:08.132908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:41569 (local address 127.2.30.129:57201)
0808 10:08:08.133060 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:08.133073 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:08.133133 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:08.133372 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:08.133373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:08.133375 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:08.133497 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:08.133500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.134528 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.134530 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:08.136007 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:08.136011 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.136888 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.136889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:08.136979 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:08.136985 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:08:08.137059 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 10:08:08.137114 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":12,"thread_start_us":81,"threads_started":1}
I20260808 10:08:08.137329 10575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:08.133130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:36139 (local address 127.2.30.129:49355)
0808 10:08:08.133302 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:08.133312 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:08.133371 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:08.133519 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:08.133520 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:08.133521 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:08.133586 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:08.133587 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.134533 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.134533 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:08.136007 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:08.136011 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.136890 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.136890 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:08.136983 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:08.136987 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:08:08.137059 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 10:08:08.137114 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":141,"thread_start_us":46,"threads_started":1}
I20260808 10:08:08.137519 10574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:08.133006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:57201 (local address 127.2.30.130:41569)
0808 10:08:08.133167 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:08.133170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:08.133181 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:08:08.133216 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:08.133217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:08.133273 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:08.133343 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:08.133581 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.134397 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.136178 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.136774 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.136850 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:08.136860 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:08.137101 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 10:08:08.137168 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":81,"threads_started":1}
I20260808 10:08:08.137598 10400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48d488da66b74bfcaed1664f103f855b" candidate_uuid: "b4c570f847b34048861611cc7eb2bc5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c4b255262bf4f5597302c241f2db53e" is_pre_election: true
I20260808 10:08:08.137770 10400 raft_consensus.cc:2454] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4c570f847b34048861611cc7eb2bc5b in term 0.
I20260808 10:08:08.137806 10255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48d488da66b74bfcaed1664f103f855b" candidate_uuid: "b4c570f847b34048861611cc7eb2bc5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efbe6fa7b3704e8a84a2638603210731" is_pre_election: true
I20260808 10:08:08.137930 10255 raft_consensus.cc:2454] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4c570f847b34048861611cc7eb2bc5b in term 0.
I20260808 10:08:08.138012 10042 leader_election.cc:302] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c4b255262bf4f5597302c241f2db53e, b4c570f847b34048861611cc7eb2bc5b; no voters:
I20260808 10:08:08.138188 10484 raft_consensus.cc:2781] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:08:08.138249 10484 raft_consensus.cc:482] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:08:08.138278 10484 raft_consensus.cc:3037] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:08.139053 10484 raft_consensus.cc:504] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:08.139204 10484 leader_election.cc:288] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [CANDIDATE]: Term 1 election: Requested vote from peers efbe6fa7b3704e8a84a2638603210731 (127.2.30.130:41569), 7c4b255262bf4f5597302c241f2db53e (127.2.30.131:36139)
I20260808 10:08:08.139400 10400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48d488da66b74bfcaed1664f103f855b" candidate_uuid: "b4c570f847b34048861611cc7eb2bc5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c4b255262bf4f5597302c241f2db53e"
I20260808 10:08:08.139400 10255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48d488da66b74bfcaed1664f103f855b" candidate_uuid: "b4c570f847b34048861611cc7eb2bc5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efbe6fa7b3704e8a84a2638603210731"
I20260808 10:08:08.139493 10255 raft_consensus.cc:3037] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:08.139493 10400 raft_consensus.cc:3037] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:08.140348 10255 raft_consensus.cc:2454] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4c570f847b34048861611cc7eb2bc5b in term 1.
I20260808 10:08:08.140359 10400 raft_consensus.cc:2454] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4c570f847b34048861611cc7eb2bc5b in term 1.
I20260808 10:08:08.140618 10042 leader_election.cc:302] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c4b255262bf4f5597302c241f2db53e, b4c570f847b34048861611cc7eb2bc5b; no voters:
I20260808 10:08:08.140722 10484 raft_consensus.cc:2781] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:08:08.140929 10484 raft_consensus.cc:686] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [term 1 LEADER]: Becoming Leader. State: Replica: b4c570f847b34048861611cc7eb2bc5b, State: Running, Role: LEADER
I20260808 10:08:08.141026 10484 consensus_queue.cc:237] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } }
I20260808 10:08:08.141983 9947 catalog_manager.cc:5495] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b reported cstate change: term changed from 0 to 1, leader changed from <none> to b4c570f847b34048861611cc7eb2bc5b (127.2.30.129). New cstate: current_term: 1 leader_uuid: "b4c570f847b34048861611cc7eb2bc5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4c570f847b34048861611cc7eb2bc5b" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 } health_report { overall_health: UNKNOWN } } }
I20260808 10:08:08.164669 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:08.161884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38210 (local address 127.2.30.129:45649)
0808 10:08:08.162068 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:08.162070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:08.162076 (+ 6us) server_negotiation.cc:407] Connection header received
0808 10:08:08.162116 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:08.162117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:08.162138 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:08.162237 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:08:08.162437 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.163031 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.163838 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.164066 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.164131 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:08.164256 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:08:08.164412 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:08:08.164474 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:08.164567 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 10:08:08.164589 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":99,"threads_started":1}
I20260808 10:08:08.170750 10476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:08.168136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:52253 (local address 127.2.30.131:36139)
0808 10:08:08.168197 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:08.168199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:08.168244 (+ 45us) server_negotiation.cc:407] Connection header received
0808 10:08:08.168266 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:08.168267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:08.168277 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:08.168323 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:08:08.168547 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.169094 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.169964 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.170152 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.170219 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:08.170492 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:08:08.170535 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:08:08.170591 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:08.170653 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 10:08:08.170682 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:08:08.170794 10581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:08.167893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:36139 (local address 127.2.30.129:52253)
0808 10:08:08.168092 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:08.168102 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:08.168204 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:08.168364 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:08.168365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:08.168367 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:08:08.168449 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:08.168450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.169182 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.169183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:08.169878 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:08.169881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.170297 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.170299 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:08.170361 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:08.170379 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:08:08.170610 (+ 231us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:08:08.170612 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:08:08.170651 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 10:08:08.170694 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":114,"threads_started":1}
I20260808 10:08:08.173476 10400 raft_consensus.cc:1261] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e [term 1 FOLLOWER]: Refusing update from remote peer b4c570f847b34048861611cc7eb2bc5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:08:08.173447 10255 raft_consensus.cc:1261] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Refusing update from remote peer b4c570f847b34048861611cc7eb2bc5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:08:08.173761 10484 consensus_queue.cc:1035] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c4b255262bf4f5597302c241f2db53e" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:08:08.173921 10576 consensus_queue.cc:1035] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "efbe6fa7b3704e8a84a2638603210731" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:08:08.181823 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:08.179230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:51187 (local address 127.2.30.129:45649)
0808 10:08:08.179280 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:08.179282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:08.179362 (+ 80us) server_negotiation.cc:407] Connection header received
0808 10:08:08.179480 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:08.179481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:08.179491 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:08.179567 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:08:08.179733 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.180264 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.181050 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.181318 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.181394 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:08.181553 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:08:08.181587 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:08:08.181638 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:08.181742 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 10:08:08.181763 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:08:08.181888 10587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:08.179148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:45649 (local address 127.2.30.131:51187)
0808 10:08:08.179282 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:08.179294 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:08.179379 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:08.179618 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:08.179619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:08.179621 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 10:08:08.179676 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:08.179678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.180352 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.180353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:08.180960 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:08.180962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:08.181394 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:08.181394 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:08.181455 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:08.181460 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 10:08:08.181640 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 10:08:08.181642 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:08:08.181726 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:08:08.181785 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260808 10:08:08.188972 9947 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:08:08.189124 9947 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:08:08.189826 9947 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c34ab8d460aa4d8e9d5ab3dec6e36a94: Sending DeleteTablet for 3 replicas of tablet 48d488da66b74bfcaed1664f103f855b
I20260808 10:08:08.190093 10091 tablet_service.cc:1514] Processing DeleteTablet for tablet 48d488da66b74bfcaed1664f103f855b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:08 UTC) from {username='oryx', principal='oryx/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:38198
I20260808 10:08:08.190208 10548 tablet_replica.cc:330] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b: stopping tablet replica
I20260808 10:08:08.190192 10234 tablet_service.cc:1514] Processing DeleteTablet for tablet 48d488da66b74bfcaed1664f103f855b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:08 UTC) from {username='oryx', principal='oryx/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:35094
I20260808 10:08:08.190261 10548 raft_consensus.cc:2229] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:08:08.190299 10549 tablet_replica.cc:330] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731: stopping tablet replica
I20260808 10:08:08.190342 10548 raft_consensus.cc:2258] T 48d488da66b74bfcaed1664f103f855b P b4c570f847b34048861611cc7eb2bc5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:08.190356 10549 raft_consensus.cc:2229] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:08:08.190394 10549 raft_consensus.cc:2258] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:08.190442 10379 tablet_service.cc:1514] Processing DeleteTablet for tablet 48d488da66b74bfcaed1664f103f855b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:08 UTC) from {username='oryx', principal='oryx/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:39100
I20260808 10:08:08.190471 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 10024
I20260808 10:08:08.190677 10550 tablet_replica.cc:330] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e: stopping tablet replica
I20260808 10:08:08.190750 10550 raft_consensus.cc:2229] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:08:08.190795 10550 raft_consensus.cc:2258] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:08.192329 10549 ts_tablet_manager.cc:1905] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:08.193288 10550 ts_tablet_manager.cc:1905] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:08.193990 10549 ts_tablet_manager.cc:1918] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:08:08.194082 10549 log.cc:1196] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/48d488da66b74bfcaed1664f103f855b
I20260808 10:08:08.194314 10549 ts_tablet_manager.cc:1939] T 48d488da66b74bfcaed1664f103f855b P efbe6fa7b3704e8a84a2638603210731: Deleting consensus metadata
I20260808 10:08:08.194730 9932 catalog_manager.cc:4841] TS efbe6fa7b3704e8a84a2638603210731 (127.2.30.130:41569): tablet 48d488da66b74bfcaed1664f103f855b (table test-table [id=b967b03a69cd42b087b8e57e6a7f2db4]) successfully deleted
I20260808 10:08:08.195071 10550 ts_tablet_manager.cc:1918] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 10:08:08.195137 10550 log.cc:1196] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/48d488da66b74bfcaed1664f103f855b
I20260808 10:08:08.195349 10550 ts_tablet_manager.cc:1939] T 48d488da66b74bfcaed1664f103f855b P 7c4b255262bf4f5597302c241f2db53e: Deleting consensus metadata
I20260808 10:08:08.195849 9932 catalog_manager.cc:4841] TS 7c4b255262bf4f5597302c241f2db53e (127.2.30.131:36139): tablet 48d488da66b74bfcaed1664f103f855b (table test-table [id=b967b03a69cd42b087b8e57e6a7f2db4]) successfully deleted
W20260808 10:08:08.198665 9932 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:45649 (error 108)
I20260808 10:08:08.199354 10593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:08.199122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:45649 (local address 127.0.0.1:38214)
0808 10:08:08.199250 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:08.199301 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:45649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":58,"threads_started":1}
I20260808 10:08:08.199360 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 10169
W20260808 10:08:08.201110 9932 catalog_manager.cc:4584] TS b4c570f847b34048861611cc7eb2bc5b (127.2.30.129:45649): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 48d488da66b74bfcaed1664f103f855b: Network error: Client connection negotiation failed: client connection to 127.2.30.129:45649: connect: Connection refused (error 111)
I20260808 10:08:08.207309 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 10313
I20260808 10:08:08.215188 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 9914
2026-08-08T10:08:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3322 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:08:08 dist-test-slave-w206 krb5kdc[10597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:08:08 dist-test-slave-w206 krb5kdc[10597](info): set up 2 sockets
Aug 08 10:08:08 dist-test-slave-w206 krb5kdc[10597](info): commencing operation
krb5kdc: starting...
W20260808 10:08:09.328395 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:08:09 dist-test-slave-w206 krb5kdc[10597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:08:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:08:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:08:09.463186 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38259
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:38259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:09.547607 10612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:09.547928 10612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:09.547973 10612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:09.549213 10612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:08:09.549299 10612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:09.549322 10612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:09.549345 10612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:08:09.549373 10612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:08:09.551031 10612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:38259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:09.551352 10612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:09.551604 10612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:09.556978 10617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:09.556988 10618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:09.557003 10620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:09.557461 10612 server_base.cc:1034] running on GCE node
I20260808 10:08:09.557729 10612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:09.557996 10612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:09.559166 10612 hybrid_clock.cc:648] HybridClock initialized: now 1786183689559149 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T10:08:09Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1353 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:08:09 dist-test-slave-w206 krb5kdc[10625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:08:09 dist-test-slave-w206 krb5kdc[10625](info): set up 2 sockets
Aug 08 10:08:09 dist-test-slave-w206 krb5kdc[10625](info): commencing operation
krb5kdc: starting...
W20260808 10:08:10.648499 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:08:10 dist-test-slave-w206 krb5kdc[10625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:08:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:08:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:08:10.783829 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38533
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:38533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:10.871574 10638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:10.871867 10638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:10.871912 10638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:10.873260 10638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:08:10.873340 10638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:10.873364 10638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:10.873387 10638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:08:10.873441 10638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:08:10.875083 10638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:38533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:10.875418 10638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:10.875705 10638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:10.881002 10643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:10.881002 10646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:10.881029 10644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:10.881310 10638 server_base.cc:1034] running on GCE node
I20260808 10:08:10.881601 10638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:10.881904 10638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:10.883059 10638 hybrid_clock.cc:648] HybridClock initialized: now 1786183690883055 us; error 40 us; skew 500 ppm
Aug 08 10:08:10 dist-test-slave-w206 krb5kdc[10625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183690, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:10.885754 10638 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:08:10.886276 10638 webserver.cc:460] Webserver started at http://127.2.30.190:42619/ using document root <none> and password file <none>
I20260808 10:08:10.886509 10638 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:10.886576 10638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:10.886729 10638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:10.887938 10638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "74232b6f6fdd465ab2f39783ce17f2ce"
format_stamp: "Formatted at 2026-08-08 10:08:10 on dist-test-slave-w206"
server_key: "bcd47f9a49d647a3adb0ff5f3da7d415"
server_key_iv: "110ce593f2beac79ab83cedcfc4f11c3"
server_key_version: "encryptionkey@0"
I20260808 10:08:10.888322 10638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "74232b6f6fdd465ab2f39783ce17f2ce"
format_stamp: "Formatted at 2026-08-08 10:08:10 on dist-test-slave-w206"
server_key: "bcd47f9a49d647a3adb0ff5f3da7d415"
server_key_iv: "110ce593f2beac79ab83cedcfc4f11c3"
server_key_version: "encryptionkey@0"
I20260808 10:08:10.889787 10638 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:08:10.890663 10653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:10.890890 10638 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:10.891024 10638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "74232b6f6fdd465ab2f39783ce17f2ce"
format_stamp: "Formatted at 2026-08-08 10:08:10 on dist-test-slave-w206"
server_key: "bcd47f9a49d647a3adb0ff5f3da7d415"
server_key_iv: "110ce593f2beac79ab83cedcfc4f11c3"
server_key_version: "encryptionkey@0"
I20260808 10:08:10.891104 10638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:10.933768 10638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:10.936362 10638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:10.936585 10638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:10.941022 10638 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:38533
I20260808 10:08:10.941087 10707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:38533 every 8 connection(s)
I20260808 10:08:10.941480 10638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:08:10.942399 10708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:10.944933 10708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce: Bootstrap starting.
I20260808 10:08:10.945488 10708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:10.945744 10708 log.cc:824] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:10.946484 10708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce: No bootstrap required, opened a new log
I20260808 10:08:10.948345 10708 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74232b6f6fdd465ab2f39783ce17f2ce" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38533 } }
I20260808 10:08:10.948578 10708 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:10.948606 10708 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74232b6f6fdd465ab2f39783ce17f2ce, State: Initialized, Role: FOLLOWER
I20260808 10:08:10.948783 10708 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74232b6f6fdd465ab2f39783ce17f2ce" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38533 } }
I20260808 10:08:10.948877 10708 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:08:10.948905 10708 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:08:10.948940 10708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:10.949581 10708 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74232b6f6fdd465ab2f39783ce17f2ce" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38533 } }
I20260808 10:08:10.949698 10708 leader_election.cc:302] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74232b6f6fdd465ab2f39783ce17f2ce; no voters:
I20260808 10:08:10.949880 10708 leader_election.cc:288] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:08:10.949962 10711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:08:10.950235 10708 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:08:10.950285 10711 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 1 LEADER]: Becoming Leader. State: Replica: 74232b6f6fdd465ab2f39783ce17f2ce, State: Running, Role: LEADER
I20260808 10:08:10.950628 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 10638
I20260808 10:08:10.950778 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:08:10.951045 2170 external_mini_cluster.cc:1428] Setting key 96fe55b063fc6d89879ad575178dfe3f
I20260808 10:08:10.951437 10711 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74232b6f6fdd465ab2f39783ce17f2ce" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38533 } }
I20260808 10:08:10.952112 10712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74232b6f6fdd465ab2f39783ce17f2ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74232b6f6fdd465ab2f39783ce17f2ce" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38533 } } }
I20260808 10:08:10.952234 10712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:10.952497 10713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74232b6f6fdd465ab2f39783ce17f2ce. Latest consensus state: current_term: 1 leader_uuid: "74232b6f6fdd465ab2f39783ce17f2ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74232b6f6fdd465ab2f39783ce17f2ce" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38533 } } }
I20260808 10:08:10.953106 10713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [sys.catalog]: This master's current role is: LEADER
W20260808 10:08:10.953275 10729 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:08:10.953352 10729 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:08:10.953435 10728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:08:10.953601 10728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:08:10.955189 10728 catalog_manager.cc:1347] Generated new cluster ID: ba36648d37514198b7dbed841de26864
I20260808 10:08:10.955263 10728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:08:10 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:08:10.962358 10728 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:08:10.963104 10728 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:08:10.963965 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:10.951669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35334 (local address 127.2.30.190:38533)
0808 10:08:10.951864 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:10.951877 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:10.951915 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:08:10.952035 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:10.952041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:10.952097 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:10.952245 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:10.953349 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:10.954772 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:10.955919 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:10.956531 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:10.956649 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:10.959708 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:10.959712 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:10.959713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:10.959726 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:10.961895 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:10.962382 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:10.962385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:10.962386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:10.962521 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:10.962754 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:10.962755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:10.962755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:10.963131 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:10.963330 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:10.963584 (+ 254us) server_negotiation.cc:299] Negotiation successful
0808 10:08:10.963756 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":133,"thread_start_us":110,"threads_started":1}
I20260808 10:08:10.971740 10728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce: Generated new TSK 0
I20260808 10:08:10.971968 10728 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:08:11.003338 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38533
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:11.094735 10735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:11.094980 10735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:11.095048 10735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:11.096241 10735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:11.096282 10735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:11.096309 10735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:08:11.097723 10735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:11.097929 10735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:11.098114 10735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:11.103513 10741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:11.103515 10743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:11.103751 10735 server_base.cc:1034] running on GCE node
W20260808 10:08:11.103561 10740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:11.104115 10735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:11.104418 10735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:11.105577 10735 hybrid_clock.cc:648] HybridClock initialized: now 1786183691105539 us; error 65 us; skew 500 ppm
Aug 08 10:08:11 dist-test-slave-w206 krb5kdc[10625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183691, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:11.108350 10735 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:08:11.108826 10735 webserver.cc:460] Webserver started at http://127.2.30.129:42567/ using document root <none> and password file <none>
I20260808 10:08:11.109092 10735 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:11.109164 10735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:11.109324 10735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:11.110487 10735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "ce007ac3093a44f48ad909bf59c22e82"
format_stamp: "Formatted at 2026-08-08 10:08:11 on dist-test-slave-w206"
server_key: "ac692ef097081a7bed1eacf70e819596"
server_key_iv: "81fbf113cb683bee8cfa32c986ee6cf6"
server_key_version: "encryptionkey@0"
I20260808 10:08:11.110857 10735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "ce007ac3093a44f48ad909bf59c22e82"
format_stamp: "Formatted at 2026-08-08 10:08:11 on dist-test-slave-w206"
server_key: "ac692ef097081a7bed1eacf70e819596"
server_key_iv: "81fbf113cb683bee8cfa32c986ee6cf6"
server_key_version: "encryptionkey@0"
I20260808 10:08:11.112344 10735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:08:11.113183 10750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:11.113453 10735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:08:11.113562 10735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "ce007ac3093a44f48ad909bf59c22e82"
format_stamp: "Formatted at 2026-08-08 10:08:11 on dist-test-slave-w206"
server_key: "ac692ef097081a7bed1eacf70e819596"
server_key_iv: "81fbf113cb683bee8cfa32c986ee6cf6"
server_key_version: "encryptionkey@0"
I20260808 10:08:11.113641 10735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:11.147116 10735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:11.149672 10735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:11.149806 10735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:11.150009 10735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:11.150426 10735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:11.150486 10735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:11.150544 10735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:11.150588 10735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:11.156774 10735 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:44505
I20260808 10:08:11.156828 10869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:44505 every 8 connection(s)
I20260808 10:08:11.157188 10735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:08:11.160351 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 10735
I20260808 10:08:11.160444 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:08:11.160653 2170 external_mini_cluster.cc:1428] Setting key 864304dabd223051c73486dd24abbfbc
Aug 08 10:08:11 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183691, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:08:11.168192 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:11.157992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:50675 (local address 127.2.30.190:38533)
0808 10:08:11.158064 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:11.158067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:11.158334 (+ 267us) server_negotiation.cc:407] Connection header received
0808 10:08:11.158809 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:11.158810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:11.158821 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:11.158897 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:11.160037 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.160612 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.161888 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.162111 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.162187 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:11.164913 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:11.164915 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:11.164915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:11.164923 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:11.166584 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:11.167097 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:11.167098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:11.167099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:11.167164 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:11.167455 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:11.167456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:11.167456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:11.167582 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:11.167656 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:11.167976 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 10:08:11.168094 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:08:11.168373 10872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:11.157979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:50675)
0808 10:08:11.158155 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:11.158226 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:11.158718 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:11.159023 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:11.159028 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:11.159695 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:11.159925 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:11.159930 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.160713 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.160714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:11.161758 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:11.161761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.162197 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.162198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:11.162328 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:11.162910 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:11.162919 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:11.164785 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:11.166684 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:11.166695 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:11.166697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:11.167027 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:11.167236 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:11.167237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:11.167237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:11.167368 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:11.167657 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:11.167666 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:11.167829 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 10:08:11.167938 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 10:08:11.168116 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":94,"threads_started":1}
I20260808 10:08:11.168953 10870 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38533
I20260808 10:08:11.169052 10870 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:11.169260 10870 heartbeater.cc:511] Master 127.2.30.190:38533 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
I20260808 10:08:11.169796 10670 ts_manager.cc:194] Registered new tserver with Master: ce007ac3093a44f48ad909bf59c22e82 (127.2.30.129:44505)
I20260808 10:08:11.170459 10670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:50675
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:08:11.190760 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38533
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:11.277629 10875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:11.277890 10875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:11.277932 10875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:11.279173 10875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:11.279256 10875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:11.279304 10875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:08:11.280865 10875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:11.281159 10875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:11.281420 10875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:11.286899 10881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:11.287014 10875 server_base.cc:1034] running on GCE node
W20260808 10:08:11.286880 10880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:11.286875 10883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:11.287402 10875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:11.287672 10875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:11.288839 10875 hybrid_clock.cc:648] HybridClock initialized: now 1786183691288817 us; error 35 us; skew 500 ppm
Aug 08 10:08:11 dist-test-slave-w206 krb5kdc[10625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183691, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:11.291559 10875 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:08:11.292025 10875 webserver.cc:460] Webserver started at http://127.2.30.130:41645/ using document root <none> and password file <none>
I20260808 10:08:11.292248 10875 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:11.292289 10875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:11.292383 10875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:11.293471 10875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "f0fdaed184734ff2bc9d28a449027379"
format_stamp: "Formatted at 2026-08-08 10:08:11 on dist-test-slave-w206"
server_key: "e39ea9512abe14a73d8cdfe2d5927822"
server_key_iv: "09a7ca72a2ab29b3367feefd5e4f6192"
server_key_version: "encryptionkey@0"
I20260808 10:08:11.293838 10875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "f0fdaed184734ff2bc9d28a449027379"
format_stamp: "Formatted at 2026-08-08 10:08:11 on dist-test-slave-w206"
server_key: "e39ea9512abe14a73d8cdfe2d5927822"
server_key_iv: "09a7ca72a2ab29b3367feefd5e4f6192"
server_key_version: "encryptionkey@0"
I20260808 10:08:11.295415 10875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:08:11.296382 10890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:11.296589 10875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:11.296677 10875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "f0fdaed184734ff2bc9d28a449027379"
format_stamp: "Formatted at 2026-08-08 10:08:11 on dist-test-slave-w206"
server_key: "e39ea9512abe14a73d8cdfe2d5927822"
server_key_iv: "09a7ca72a2ab29b3367feefd5e4f6192"
server_key_version: "encryptionkey@0"
I20260808 10:08:11.296792 10875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:11.307505 10875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:11.310009 10875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:11.310181 10875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:11.310428 10875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:11.310755 10875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:11.310809 10875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:11.310851 10875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:11.310877 10875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:11.317194 10875 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:45493
I20260808 10:08:11.317235 11010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:45493 every 8 connection(s)
I20260808 10:08:11.317658 10875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
Aug 08 10:08:11 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183691, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:08:11.327138 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 10875
I20260808 10:08:11.327277 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:08:11.327495 2170 external_mini_cluster.cc:1428] Setting key c9b4837b00943e8d17a6f5c8ffb85208
I20260808 10:08:11.327587 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:11.318459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:53435 (local address 127.2.30.190:38533)
0808 10:08:11.318530 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:11.318532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:11.318717 (+ 185us) server_negotiation.cc:407] Connection header received
0808 10:08:11.319264 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:11.319265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:11.319275 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:11.319351 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:11.320335 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.320853 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.321880 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.322062 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.322115 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:11.324687 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:11.324688 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:11.324688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:11.324695 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:11.326063 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:11.326471 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:11.326472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:11.326473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:11.326522 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:11.326750 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:11.326751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:11.326751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:11.326867 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:11.326918 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:11.327359 (+ 441us) server_negotiation.cc:299] Negotiation successful
0808 10:08:11.327449 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:08:11.327788 11013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:11.318425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:53435)
0808 10:08:11.318586 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:11.318628 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:11.319164 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:11.319406 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:11.319409 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:11.320026 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:11.320240 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:11.320245 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.320963 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.320964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:11.321781 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:11.321784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.322238 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.322239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:11.322359 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:11.323086 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:11.323095 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:11.324599 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:11.326150 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:11.326157 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:11.326158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:11.326405 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:11.326581 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:11.326582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:11.326582 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:11.326695 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:11.326924 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:11.326928 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:11.327114 (+ 186us) client_negotiation.cc:769] Sending connection context
0808 10:08:11.327322 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 10:08:11.327541 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260808 10:08:11.328378 11011 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38533
I20260808 10:08:11.328495 11011 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:11.328697 11011 heartbeater.cc:511] Master 127.2.30.190:38533 requested a full tablet report, sending...
I20260808 10:08:11.329072 10670 ts_manager.cc:194] Registered new tserver with Master: f0fdaed184734ff2bc9d28a449027379 (127.2.30.130:45493)
I20260808 10:08:11.329492 10670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:53435
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:08:11.354306 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38533
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:11.439078 11016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:11.439314 11016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:11.439405 11016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:11.441102 11016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:11.441171 11016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:11.441201 11016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:08:11.442811 11016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:11.443120 11016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:11.443361 11016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:11.448822 11024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:11.448838 11022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:11.448901 11016 server_base.cc:1034] running on GCE node
W20260808 10:08:11.448840 11021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:11.449234 11016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:11.449563 11016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:11.450743 11016 hybrid_clock.cc:648] HybridClock initialized: now 1786183691450707 us; error 46 us; skew 500 ppm
Aug 08 10:08:11 dist-test-slave-w206 krb5kdc[10625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183691, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:11.453634 11016 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:08:11.454177 11016 webserver.cc:460] Webserver started at http://127.2.30.131:39421/ using document root <none> and password file <none>
I20260808 10:08:11.454434 11016 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:11.454507 11016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:11.454660 11016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:11.455865 11016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "b0e34a5060444c959fc2d1b9954ad071"
format_stamp: "Formatted at 2026-08-08 10:08:11 on dist-test-slave-w206"
server_key: "063bf88043e3ff002eddc8585048e767"
server_key_iv: "97c1d11ea029f4b0c5c4769779228fe3"
server_key_version: "encryptionkey@0"
I20260808 10:08:11.456243 11016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "b0e34a5060444c959fc2d1b9954ad071"
format_stamp: "Formatted at 2026-08-08 10:08:11 on dist-test-slave-w206"
server_key: "063bf88043e3ff002eddc8585048e767"
server_key_iv: "97c1d11ea029f4b0c5c4769779228fe3"
server_key_version: "encryptionkey@0"
I20260808 10:08:11.457737 11016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:08:11.458611 11031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:11.458884 11016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:11.459015 11016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "b0e34a5060444c959fc2d1b9954ad071"
format_stamp: "Formatted at 2026-08-08 10:08:11 on dist-test-slave-w206"
server_key: "063bf88043e3ff002eddc8585048e767"
server_key_iv: "97c1d11ea029f4b0c5c4769779228fe3"
server_key_version: "encryptionkey@0"
I20260808 10:08:11.459108 11016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:11.467684 11016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:11.469866 11016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:11.470038 11016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:11.470270 11016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:11.470606 11016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:11.470664 11016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:11.470705 11016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:11.470731 11016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:11.477118 11016 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:38599
I20260808 10:08:11.477197 11147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:38599 every 8 connection(s)
I20260808 10:08:11.477576 11016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:08:11.480101 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 11016
I20260808 10:08:11.480224 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:08:11.480458 2170 external_mini_cluster.cc:1428] Setting key 2c11d2aa69c9d52a04f7e2727a62cd4d
Aug 08 10:08:11 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183691, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:08:11.487126 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:11.478356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:54859 (local address 127.2.30.190:38533)
0808 10:08:11.478401 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:11.478403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:11.478620 (+ 217us) server_negotiation.cc:407] Connection header received
0808 10:08:11.479171 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:11.479172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:11.479182 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:11.479244 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:11.480251 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.480904 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.481925 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.482117 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.482173 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:11.484535 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:11.484536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:11.484537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:11.484544 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:11.485854 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:11.486229 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:11.486229 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:11.486230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:11.486280 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:11.486507 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:11.486508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:11.486508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:11.486616 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:11.486670 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:11.486920 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 10:08:11.487048 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:08:11.487416 11150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:11.478316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:54859)
0808 10:08:11.478488 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:11.478531 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:11.479085 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:11.479297 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:11.479300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:11.479890 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:11.480097 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:11.480101 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.480997 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.480998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:11.481845 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:11.481848 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.482196 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.482197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:11.482356 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:11.482947 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:11.482955 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:11.484445 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:11.485933 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:11.485940 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:11.485941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:11.486161 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:11.486339 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:11.486339 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:11.486340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:11.486450 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:11.486673 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:11.486678 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:11.486805 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 10:08:11.486889 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 10:08:11.487103 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260808 10:08:11.487902 11148 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38533
I20260808 10:08:11.488016 11148 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:11.488211 11148 heartbeater.cc:511] Master 127.2.30.190:38533 requested a full tablet report, sending...
I20260808 10:08:11.488613 10670 ts_manager.cc:194] Registered new tserver with Master: b0e34a5060444c959fc2d1b9954ad071 (127.2.30.131:38599)
I20260808 10:08:11.489017 10670 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:54859
I20260808 10:08:11.492547 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.129.keytab.
Entry for principal oryx/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.129.keytab.
WARNING: no policy specified for oryx/127.2.30.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.130.keytab.
Entry for principal oryx/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.130.keytab.
WARNING: no policy specified for oryx/127.2.30.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.131.keytab.
Entry for principal oryx/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.131.keytab.
I20260808 10:08:11.580112 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 10735
I20260808 10:08:11.586700 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 10875
I20260808 10:08:11.592022 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 11016
I20260808 10:08:11.597474 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 10638
I20260808 10:08:11.604180 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38533
--webserver_interface=127.2.30.190
--webserver_port=42619
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:38533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:11.689098 11158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:11.689322 11158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:11.689347 11158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:11.690582 11158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:08:11.690629 11158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:11.690642 11158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:11.690656 11158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:08:11.690668 11158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:08:11.692273 11158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:38533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=42619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:11.692570 11158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:11.692819 11158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:11.698660 11166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:11.698665 11163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:11.698679 11164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:11.698729 11158 server_base.cc:1034] running on GCE node
I20260808 10:08:11.699251 11158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:11.699570 11158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:11.700779 11158 hybrid_clock.cc:648] HybridClock initialized: now 1786183691700761 us; error 25 us; skew 500 ppm
Aug 08 10:08:11 dist-test-slave-w206 krb5kdc[10625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183691, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:11.703563 11158 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:08:11.704002 11158 webserver.cc:460] Webserver started at http://127.2.30.190:42619/ using document root <none> and password file <none>
I20260808 10:08:11.704222 11158 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:11.704288 11158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:11.705693 11158 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:11.706457 11173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:11.706656 11158 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:11.706768 11158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "74232b6f6fdd465ab2f39783ce17f2ce"
format_stamp: "Formatted at 2026-08-08 10:08:10 on dist-test-slave-w206"
server_key: "bcd47f9a49d647a3adb0ff5f3da7d415"
server_key_iv: "110ce593f2beac79ab83cedcfc4f11c3"
server_key_version: "encryptionkey@0"
I20260808 10:08:11.707199 11158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:11.745015 11158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:11.747126 11158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:11.747306 11158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:11.751415 11158 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:38533
I20260808 10:08:11.751458 11228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:38533 every 8 connection(s)
I20260808 10:08:11.751884 11158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:08:11.753482 11229 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:08:11.754292 11229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce: Bootstrap starting.
I20260808 10:08:11.760154 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 11158
I20260808 10:08:11.760646 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:44505
--local_ip_for_outbound_sockets=127.2.30.129
--tserver_master_addrs=127.2.30.190:38533
--webserver_port=42567
--webserver_interface=127.2.30.129
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.30.129
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:08:11.763761 11229 log.cc:824] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:11.765653 11229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:08:11.765972 11229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce: Bootstrap complete.
I20260808 10:08:11.768392 11229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74232b6f6fdd465ab2f39783ce17f2ce" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38533 } }
I20260808 10:08:11.768616 11229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74232b6f6fdd465ab2f39783ce17f2ce, State: Initialized, Role: FOLLOWER
I20260808 10:08:11.768781 11229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74232b6f6fdd465ab2f39783ce17f2ce" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38533 } }
I20260808 10:08:11.768879 11229 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:08:11.768955 11229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:08:11.769011 11229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:08:11.770036 11229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74232b6f6fdd465ab2f39783ce17f2ce" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38533 } }
I20260808 10:08:11.770157 11229 leader_election.cc:302] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74232b6f6fdd465ab2f39783ce17f2ce; no voters:
I20260808 10:08:11.770394 11229 leader_election.cc:288] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:08:11.770432 11233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:08:11.770551 11233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [term 2 LEADER]: Becoming Leader. State: Replica: 74232b6f6fdd465ab2f39783ce17f2ce, State: Running, Role: LEADER
I20260808 10:08:11.770641 11233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74232b6f6fdd465ab2f39783ce17f2ce" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38533 } }
I20260808 10:08:11.770673 11229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:08:11.770967 11235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "74232b6f6fdd465ab2f39783ce17f2ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74232b6f6fdd465ab2f39783ce17f2ce" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38533 } } }
I20260808 10:08:11.771097 11235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:11.771397 11234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74232b6f6fdd465ab2f39783ce17f2ce. Latest consensus state: current_term: 2 leader_uuid: "74232b6f6fdd465ab2f39783ce17f2ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74232b6f6fdd465ab2f39783ce17f2ce" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38533 } } }
I20260808 10:08:11.771528 11234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:11.771620 11239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:08:11.772190 11239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:08:11.772356 11239 catalog_manager.cc:1259] Loaded cluster ID: ba36648d37514198b7dbed841de26864
I20260808 10:08:11.772400 11239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:08:11.773257 11239 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:08:11.773465 11239 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 74232b6f6fdd465ab2f39783ce17f2ce: Loaded TSK: 0
I20260808 10:08:11.773689 11239 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:08:11.862387 11231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:11.862637 11231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:11.862680 11231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:11.863958 11231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:11.864030 11231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:11.864094 11231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:08:11.865607 11231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.129.keytab
--principal=oryx/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:44505
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=42567
--tserver_master_addrs=127.2.30.190:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:11.865885 11231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:11.866127 11231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:11.871315 11257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:11.871421 11258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:11.871622 11231 server_base.cc:1034] running on GCE node
W20260808 10:08:11.871313 11260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:11.872064 11231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:11.872375 11231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:11.873541 11231 hybrid_clock.cc:648] HybridClock initialized: now 1786183691873529 us; error 30 us; skew 500 ppm
Aug 08 10:08:11 dist-test-slave-w206 krb5kdc[10625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183691, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:11.876513 11231 init.cc:377] Logged in from keytab as oryx/127.2.30.129@KRBTEST.COM (short username oryx)
I20260808 10:08:11.877067 11231 webserver.cc:460] Webserver started at http://127.2.30.129:42567/ using document root <none> and password file <none>
I20260808 10:08:11.877381 11231 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:11.877465 11231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:11.879345 11231 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 10:08:11.880051 11267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:11.880281 11231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:08:11.880388 11231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "ce007ac3093a44f48ad909bf59c22e82"
format_stamp: "Formatted at 2026-08-08 10:08:11 on dist-test-slave-w206"
server_key: "ac692ef097081a7bed1eacf70e819596"
server_key_iv: "81fbf113cb683bee8cfa32c986ee6cf6"
server_key_version: "encryptionkey@0"
I20260808 10:08:11.880861 11231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:11.891024 11231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:11.893817 11231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:11.893985 11231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:11.894285 11231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:11.894670 11231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:11.894721 11231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:11.894753 11231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:11.894783 11231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:11.902390 11231 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:44505
I20260808 10:08:11.902486 11386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:44505 every 8 connection(s)
I20260808 10:08:11.902858 11231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:08:11.907330 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 11231
I20260808 10:08:11.907682 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:45493
--local_ip_for_outbound_sockets=127.2.30.130
--tserver_master_addrs=127.2.30.190:38533
--webserver_port=41645
--webserver_interface=127.2.30.130
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.30.130
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:08:11 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:11 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:11.913757 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:11.903783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:40021)
0808 10:08:11.903988 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:11.904041 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:11.904614 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:11.905112 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:11.905116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:11.905771 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:11.905972 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:11.905978 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.907419 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.907421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:11.908681 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:11.908685 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.909641 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.909642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:11.909795 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:11.910513 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:11.910522 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:11.913545 (+ 3023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":94,"threads_started":1}
W20260808 10:08:11.914038 11390 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:11.903860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:40021 (local address 127.2.30.190:38533)
0808 10:08:11.904091 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:11.904099 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:11.904145 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:08:11.904856 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:11.904865 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:11.904920 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:11.905064 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:11.906110 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.907294 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.908792 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.909547 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.909672 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:11.913767 (+ 4095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:40021: BlockingRecv error: recv got EOF from 127.2.30.129:40021 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":110,"threads_started":1}
W20260808 10:08:11.914260 11387 heartbeater.cc:644] Failed to heartbeat to 127.2.30.190:38533 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.30.190:38533: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Aug 08 10:08:11 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:11 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:11.921034 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:11.914549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:44325)
0808 10:08:11.914681 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:11.914693 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:11.914765 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:11.914944 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:11.914945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:11.915161 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:11.915243 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:11.915245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.916165 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.916167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:11.917194 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:11.917197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.917731 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.917733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:11.917821 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:11.918359 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:11.918366 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:11.920917 (+ 2551us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109}
W20260808 10:08:11.921294 11390 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:11.914625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:44325 (local address 127.2.30.190:38533)
0808 10:08:11.914686 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:11.914688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:11.914746 (+ 58us) server_negotiation.cc:407] Connection header received
0808 10:08:11.914837 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:11.914838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:11.914852 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:11.914921 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:11.915337 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.916070 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.917301 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.917627 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.917708 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:11.921040 (+ 3332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:44325: BlockingRecv error: recv got EOF from 127.2.30.129:44325 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 10:08:11 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:11 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:11.927245 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:11.921343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:42719)
0808 10:08:11.921596 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:11.921608 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:11.921663 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:11.921872 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:11.921873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:11.922033 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:11.922074 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:11.922075 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.923040 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.923041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:11.923959 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:11.923961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.924435 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.924436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:11.924499 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:11.924994 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:11.924999 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:11.927173 (+ 2174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":93,"threads_started":1}
W20260808 10:08:11.927609 11387 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:08:11.927659 11390 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:11.921540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:42719 (local address 127.2.30.190:38533)
0808 10:08:11.921603 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:11.921605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:11.921661 (+ 56us) server_negotiation.cc:407] Connection header received
0808 10:08:11.921728 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:11.921729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:11.921742 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:11.921839 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:11.922158 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.922940 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.924049 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:11.924357 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:11.924423 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:11.927603 (+ 3180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:42719: BlockingRecv error: recv got EOF from 127.2.30.129:42719 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 10:08:12.005590 11391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:12.005834 11391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:12.005863 11391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:12.007138 11391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:12.007177 11391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:12.007205 11391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:08:12.008745 11391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.130.keytab
--principal=oryx/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:45493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=41645
--tserver_master_addrs=127.2.30.190:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:12.008970 11391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:12.009181 11391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:12.014977 11397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:12.014995 11401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:12.015046 11398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:12.015072 11391 server_base.cc:1034] running on GCE node
I20260808 10:08:12.015478 11391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:12.015700 11391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:12.016867 11391 hybrid_clock.cc:648] HybridClock initialized: now 1786183692016851 us; error 30 us; skew 500 ppm
Aug 08 10:08:12 dist-test-slave-w206 krb5kdc[10625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183692, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:12.019744 11391 init.cc:377] Logged in from keytab as oryx/127.2.30.130@KRBTEST.COM (short username oryx)
I20260808 10:08:12.020287 11391 webserver.cc:460] Webserver started at http://127.2.30.130:41645/ using document root <none> and password file <none>
I20260808 10:08:12.020522 11391 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:12.020583 11391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:12.022012 11391 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:12.022642 11408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:12.022854 11391 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:12.022941 11391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "f0fdaed184734ff2bc9d28a449027379"
format_stamp: "Formatted at 2026-08-08 10:08:11 on dist-test-slave-w206"
server_key: "e39ea9512abe14a73d8cdfe2d5927822"
server_key_iv: "09a7ca72a2ab29b3367feefd5e4f6192"
server_key_version: "encryptionkey@0"
I20260808 10:08:12.023334 11391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:12.033974 11391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:12.036329 11391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:12.036446 11391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:12.036633 11391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:12.036978 11391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:12.037019 11391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:12.037042 11391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:12.037055 11391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:12.043107 11391 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:45493
I20260808 10:08:12.043262 11524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:45493 every 8 connection(s)
I20260808 10:08:12.043558 11391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:08:12.044723 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 11391
I20260808 10:08:12.045014 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:38599
--local_ip_for_outbound_sockets=127.2.30.131
--tserver_master_addrs=127.2.30.190:38533
--webserver_port=39421
--webserver_interface=127.2.30.131
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:35937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.30.131
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:08:12 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:12 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:12.052125 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:12.044284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:54757)
0808 10:08:12.044461 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:12.044497 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:12.045177 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:12.045434 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:12.045439 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:12.046038 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:12.046245 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:12.046257 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.047371 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.047372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:12.048182 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:12.048185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.048743 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.048744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:12.048865 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:12.049459 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:12.049469 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:12.051958 (+ 2489us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
W20260808 10:08:12.052206 11390 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:12.044301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:54757 (local address 127.2.30.190:38533)
0808 10:08:12.044370 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:12.044372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:12.044599 (+ 227us) server_negotiation.cc:407] Connection header received
0808 10:08:12.045279 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:12.045281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:12.045298 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:12.045357 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:12.046354 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.047264 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.048281 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.048626 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.048703 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:12.052103 (+ 3400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:54757: BlockingRecv error: recv got EOF from 127.2.30.130:54757 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 10:08:12.052587 11525 heartbeater.cc:644] Failed to heartbeat to 127.2.30.190:38533 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.30.190:38533: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Aug 08 10:08:12 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:12 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:12.057143 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:12.052825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:48487)
0808 10:08:12.052881 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:12.052891 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:12.052958 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:12.053073 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:12.053073 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:12.053212 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:12.053257 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:12.053259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.053891 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.053892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:12.054384 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:12.054386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.054755 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.054755 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:12.054835 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:12.055301 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:12.055306 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:12.057094 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 10:08:12.057301 11390 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:12.052899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:48487 (local address 127.2.30.190:38533)
0808 10:08:12.052968 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:12.052970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:12.052982 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:08:12.053010 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:12.053011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:12.053021 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:12.053072 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:12.053339 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.053811 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.054470 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.054671 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.054754 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:12.057207 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:48487: BlockingRecv error: recv got EOF from 127.2.30.130:48487 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 10:08:12 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:12 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:12.062125 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:12.057508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:43903)
0808 10:08:12.057618 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:12.057629 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:12.057667 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:12.057803 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:12.057804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:12.057914 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:12.057941 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:12.057942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.058852 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.058852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:12.059383 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:12.059385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.059836 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.059837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:12.059874 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:12.060358 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:12.060362 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:12.062074 (+ 1712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93}
W20260808 10:08:12.062259 11390 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:12.057620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:43903 (local address 127.2.30.190:38533)
0808 10:08:12.057676 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:12.057678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:12.057690 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:08:12.057729 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:12.057730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:12.057740 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:12.057792 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:12.058006 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.058774 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.059455 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.059755 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.059830 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:12.062191 (+ 2361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:43903: BlockingRecv error: recv got EOF from 127.2.30.130:43903 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 10:08:12.062351 11525 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:08:12.134951 11528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:12.135207 11528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:12.135234 11528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:12.136519 11528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:12.136572 11528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:12.136600 11528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:08:12.138134 11528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/krb5kdc/oryx_127.2.30.131.keytab
--principal=oryx/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:38599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=39421
--tserver_master_addrs=127.2.30.190:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:12.138415 11528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:12.138682 11528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:12.144025 11533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:12.143949 11534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:12.144141 11528 server_base.cc:1034] running on GCE node
W20260808 10:08:12.143949 11536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:12.144555 11528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:12.144821 11528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:12.145999 11528 hybrid_clock.cc:648] HybridClock initialized: now 1786183692145995 us; error 38 us; skew 500 ppm
Aug 08 10:08:12 dist-test-slave-w206 krb5kdc[10625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183692, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:12.148762 11528 init.cc:377] Logged in from keytab as oryx/127.2.30.131@KRBTEST.COM (short username oryx)
I20260808 10:08:12.149330 11528 webserver.cc:460] Webserver started at http://127.2.30.131:39421/ using document root <none> and password file <none>
I20260808 10:08:12.149585 11528 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:12.149669 11528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:12.151333 11528 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:08:12.151944 11543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:12.152156 11528 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:08:12.152272 11528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "b0e34a5060444c959fc2d1b9954ad071"
format_stamp: "Formatted at 2026-08-08 10:08:11 on dist-test-slave-w206"
server_key: "063bf88043e3ff002eddc8585048e767"
server_key_iv: "97c1d11ea029f4b0c5c4769779228fe3"
server_key_version: "encryptionkey@0"
I20260808 10:08:12.152630 11528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:12.165043 11528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:12.167553 11528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:12.167735 11528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:12.167999 11528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:12.168330 11528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:12.168394 11528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:12.168440 11528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:12.168465 11528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:12.174710 11528 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:38599
I20260808 10:08:12.174760 11661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:38599 every 8 connection(s)
I20260808 10:08:12.175181 11528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
Aug 08 10:08:12 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:12.181320 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 11528
Aug 08 10:08:12 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 10:08:12.183058 11390 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:12.175983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:59157 (local address 127.2.30.190:38533)
0808 10:08:12.176054 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:12.176056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:12.176279 (+ 223us) server_negotiation.cc:407] Connection header received
0808 10:08:12.176836 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:12.176838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:12.176851 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:12.176910 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:12.177878 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.178392 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.179494 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.179699 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.179787 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:12.182941 (+ 3154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:59157: BlockingRecv error: recv got EOF from 127.2.30.131:59157 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:08:12.183110 11664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:12.175975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:59157)
0808 10:08:12.176149 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:12.176186 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:12.176715 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:12.176984 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:12.176988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:12.177571 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:12.177767 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:12.177771 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.178516 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.178517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:12.179402 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:12.179404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.179800 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.179801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:12.179953 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:12.180428 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:12.180436 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:12.182839 (+ 2403us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
W20260808 10:08:12.183372 11662 heartbeater.cc:644] Failed to heartbeat to 127.2.30.190:38533 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.30.190:38533: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Aug 08 10:08:12 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:12 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:12.188606 11664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:12.183621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:40467)
0808 10:08:12.183671 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:12.183683 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:12.183722 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:12.183842 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:12.183843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:12.183957 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:12.184001 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:12.184002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.184877 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.184878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:12.185485 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:12.185487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.185966 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.185967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:12.186024 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:12.186412 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:12.186417 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:12.188512 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 10:08:12.188660 11390 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:12.183679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:40467 (local address 127.2.30.190:38533)
0808 10:08:12.183721 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:12.183723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:12.183732 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:08:12.183771 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:12.183771 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:12.183781 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:12.183838 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:12.184075 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.184802 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.185607 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.185881 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.185956 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:12.188593 (+ 2637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:40467: BlockingRecv error: recv got EOF from 127.2.30.131:40467 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:08:12.190266 11666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:12.181931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35348 (local address 127.2.30.190:38533)
0808 10:08:12.182045 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:12.182047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:12.182155 (+ 108us) server_negotiation.cc:407] Connection header received
0808 10:08:12.182209 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:12.182211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:12.182226 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:12.182306 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:12.182906 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.183608 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.184753 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.185026 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.185101 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:12.186628 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:12.186633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:12.186633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:12.186644 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:12.188702 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:12.189172 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:12.189176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:12.189176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:12.189289 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:12.189516 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:12.189517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:12.189517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:12.189769 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:12.189941 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:12.190041 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 10:08:12.190154 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":67,"threads_started":1}
Aug 08 10:08:12 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:12 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:12.193145 11664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:12.188925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:52655)
0808 10:08:12.188976 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:12.188989 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:12.189032 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:12.189179 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:12.189180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:12.189305 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:12.189335 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:12.189336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.189970 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.189971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:12.190645 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:12.190647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.191096 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.191097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:12.191152 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:12.191495 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:12.191499 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:12.193061 (+ 1562us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 10:08:12.193192 11390 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:12.189005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:52655 (local address 127.2.30.190:38533)
0808 10:08:12.189068 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:12.189070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:12.189078 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:08:12.189098 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:12.189098 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:12.189107 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:12.189173 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:12.189412 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.189889 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.190726 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.191038 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.191097 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:12.193130 (+ 2033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:52655: BlockingRecv error: recv got EOF from 127.2.30.131:52655 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 10:08:12.193276 11662 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 10:08:12 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:12 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:12.933676 11667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:12.928059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:57439)
0808 10:08:12.928229 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:12.928241 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:12.928303 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:12.928510 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:12.928511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:12.928689 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:12.928742 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:12.928744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.929394 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.929395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:12.930233 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:12.930237 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.930741 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.930742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:12.930787 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:12.931443 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:12.931448 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:12.933602 (+ 2154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
W20260808 10:08:12.933838 11668 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:12.928159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:57439 (local address 127.2.30.190:38533)
0808 10:08:12.928300 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:12.928302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:12.928313 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:08:12.928411 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:12.928412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:12.928424 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:12.928487 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:12.928808 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.929317 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.930310 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:12.930658 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:12.930724 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:12.933735 (+ 3011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:57439: BlockingRecv error: recv got EOF from 127.2.30.129:57439 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
Aug 08 10:08:13 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:13 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:13.067677 11669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:13.062784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:42505)
0808 10:08:13.062926 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:13.062938 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:13.063027 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:13.063148 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:13.063149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:13.063365 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:13.063417 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:13.063420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:13.064124 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:13.064125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:13.064672 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:13.064674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:13.065036 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:13.065037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:13.065079 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:13.065562 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:13.065567 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:13.067602 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
W20260808 10:08:13.067790 11668 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:13.062853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:42505 (local address 127.2.30.190:38533)
0808 10:08:13.062943 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:13.062945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:13.063006 (+ 61us) server_negotiation.cc:407] Connection header received
0808 10:08:13.063077 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:13.063078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:13.063090 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:13.063135 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:13.063491 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:13.064005 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:13.064767 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:13.064963 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:13.065028 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:13.067713 (+ 2685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:42505: BlockingRecv error: recv got EOF from 127.2.30.130:42505 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 10:08:13 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:13 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:13.198717 11670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:13.193731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:60901)
0808 10:08:13.193875 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:13.193894 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:13.193956 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:13.194097 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:13.194098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:13.194269 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:13.194329 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:13.194330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:13.195180 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:13.195181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:13.195895 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:13.195898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:13.196361 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:13.196362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:13.196406 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:13.196857 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:13.196861 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:13.198642 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":98,"threads_started":1}
W20260808 10:08:13.198860 11668 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:13.193796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:60901 (local address 127.2.30.190:38533)
0808 10:08:13.193891 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:13.193893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:13.193962 (+ 69us) server_negotiation.cc:407] Connection header received
0808 10:08:13.194005 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:13.194006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:13.194016 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:13.194081 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:13.194394 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:13.195092 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:13.196060 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:13.196276 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:13.196342 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:13.198765 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:60901: BlockingRecv error: recv got EOF from 127.2.30.131:60901 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 10:08:13 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:13 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:13.939357 11671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:13.934401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:33193)
0808 10:08:13.934570 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:13.934585 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:13.934650 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:13.934817 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:13.934818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:13.935066 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:13.935117 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:13.935119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:13.935832 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:13.935833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:13.936362 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:13.936364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:13.936749 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:13.936750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:13.936819 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:13.937320 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:13.937324 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:13.939276 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":90,"threads_started":1}
W20260808 10:08:13.939487 11672 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:13.934489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:33193 (local address 127.2.30.190:38533)
0808 10:08:13.934655 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:13.934657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:13.934670 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:08:13.934705 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:13.934705 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:13.934716 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:13.934799 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:13.935203 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:13.935710 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:13.936449 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:13.936652 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:13.936748 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:13.939411 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:33193: BlockingRecv error: recv got EOF from 127.2.30.129:33193 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
Aug 08 10:08:14 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:14 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:14.073344 11673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:14.068188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:37735)
0808 10:08:14.068407 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:14.068418 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:14.068481 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:14.068608 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:14.068608 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:14.068774 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:14.068830 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:14.068832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:14.069532 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:14.069533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:14.070089 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:14.070091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:14.070528 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:14.070529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:14.070586 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:14.071150 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:14.071155 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:14.073220 (+ 2065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":162,"threads_started":1}
W20260808 10:08:14.073397 11672 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:14.068261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:37735 (local address 127.2.30.190:38533)
0808 10:08:14.068352 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:14.068354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:14.068481 (+ 127us) server_negotiation.cc:407] Connection header received
0808 10:08:14.068538 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:14.068539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:14.068550 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:14.068603 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:14.068916 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:14.069450 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:14.070216 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:14.070438 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:14.070526 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:14.073317 (+ 2791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:37735: BlockingRecv error: recv got EOF from 127.2.30.130:37735 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 10:08:14 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:14 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:14.204491 11674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:14.199349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:43439)
0808 10:08:14.199514 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:14.199529 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:14.199616 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:14.199801 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:14.199802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:14.200006 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:14.200074 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:14.200076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:14.200765 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:14.200766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:14.201294 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:14.201295 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:14.201670 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:14.201671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:14.201741 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:14.202286 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:14.202290 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:14.204388 (+ 2098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":89,"threads_started":1}
W20260808 10:08:14.204559 11672 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:14.199504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:43439 (local address 127.2.30.190:38533)
0808 10:08:14.199551 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:14.199553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:14.199593 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:08:14.199694 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:14.199695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:14.199713 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:14.199787 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:14.200162 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:14.200671 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:14.201383 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:14.201581 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:14.201669 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:14.204493 (+ 2824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:43439: BlockingRecv error: recv got EOF from 127.2.30.131:43439 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 10:08:14 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:14 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:14.950138 11675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:14.939980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:46369)
0808 10:08:14.940192 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:14.940205 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:14.940308 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:14.940479 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:14.940480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:14.940670 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:14.940722 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:14.940724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:14.941413 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:14.941414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:14.942036 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:14.942038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:14.942446 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:14.942447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:14.942491 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:14.947870 (+ 5379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:14.947876 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:14.950036 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":103,"threads_started":1}
W20260808 10:08:14.950251 11676 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:14.940117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:46369 (local address 127.2.30.190:38533)
0808 10:08:14.940254 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:14.940256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:14.940267 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:08:14.940379 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:14.940380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:14.940390 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:14.940442 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:14.940798 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:14.941325 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:14.942120 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:14.942374 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:14.942430 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:14.950171 (+ 7741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:46369: BlockingRecv error: recv got EOF from 127.2.30.129:46369 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
Aug 08 10:08:15 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:15 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:15.078778 11677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:15.073919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:56717)
0808 10:08:15.074074 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:15.074092 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:15.074158 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:15.074310 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:15.074311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:15.074491 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:15.074544 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:15.074546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:15.075288 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:15.075289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:15.075912 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:15.075914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:15.076281 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:15.076282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:15.076334 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:15.076831 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:15.076836 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:15.078715 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
W20260808 10:08:15.078939 11676 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:15.073981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:56717 (local address 127.2.30.190:38533)
0808 10:08:15.074046 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:15.074049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:15.074155 (+ 106us) server_negotiation.cc:407] Connection header received
0808 10:08:15.074214 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:15.074214 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:15.074225 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:15.074287 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:15.074610 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:15.075189 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:15.075992 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:15.076176 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:15.076231 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:15.078859 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:56717: BlockingRecv error: recv got EOF from 127.2.30.130:56717 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 10:08:15 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:15 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:15.210206 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:15.205193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:43395)
0808 10:08:15.205356 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:15.205372 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:15.205457 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:15.205611 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:15.205612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:15.205791 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:15.205843 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:15.205844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:15.206544 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:15.206544 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:15.207121 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:15.207123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:15.207518 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:15.207519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:15.207562 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:15.208034 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:15.208039 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:15.210130 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":100,"threads_started":1}
W20260808 10:08:15.210371 11676 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:15.205308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:43395 (local address 127.2.30.190:38533)
0808 10:08:15.205442 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:15.205444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:15.205458 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:08:15.205510 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:15.205511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:15.205524 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:15.205593 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:15.205949 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:15.206466 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:15.207215 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:15.207443 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:15.207510 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:15.210230 (+ 2720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:43395: BlockingRecv error: recv got EOF from 127.2.30.131:43395 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 10:08:15 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:15 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:15.956348 11681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:15.950925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:37619)
0808 10:08:15.951115 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:15.951137 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:15.951284 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:15.951479 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:15.951481 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:15.951686 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:15.951760 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:15.951762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:15.952425 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:15.952426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:15.953044 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:15.953046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:15.953476 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:15.953477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:15.953531 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:15.954071 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:15.954076 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:15.956264 (+ 2188us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":95,"threads_started":1}
W20260808 10:08:15.956459 11680 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:15.950886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:37619 (local address 127.2.30.190:38533)
0808 10:08:15.951093 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:15.951109 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:15.951260 (+ 151us) server_negotiation.cc:407] Connection header received
0808 10:08:15.951363 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:15.951364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:15.951378 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:15.951460 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:15.951829 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:15.952346 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:15.953155 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:15.953373 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:15.953450 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:15.956385 (+ 2935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:37619: BlockingRecv error: recv got EOF from 127.2.30.129:37619 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":123,"threads_started":1}
Aug 08 10:08:16 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:16 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:16.085788 11682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:16.079374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:38783)
0808 10:08:16.079551 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:16.079563 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:16.079629 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:16.079782 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:16.079783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:16.079981 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:16.080036 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:16.080038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:16.080804 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:16.080805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:16.081399 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:16.081401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:16.081782 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:16.081784 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:16.081831 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:16.082384 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:16.082390 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:16.085681 (+ 3291us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":115,"threads_started":1}
W20260808 10:08:16.085919 11680 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:16.079470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:38783 (local address 127.2.30.190:38533)
0808 10:08:16.079556 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:16.079559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:16.079631 (+ 72us) server_negotiation.cc:407] Connection header received
0808 10:08:16.079683 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:16.079684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:16.079696 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:16.079760 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:16.080108 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:16.080708 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:16.081493 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:16.081708 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:16.081770 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:16.085847 (+ 4077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:38783: BlockingRecv error: recv got EOF from 127.2.30.130:38783 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 10:08:16 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:16 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:16.215813 11683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:16.210723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:53221)
0808 10:08:16.210881 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:16.210896 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:16.210955 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:16.211159 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:16.211160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:16.211339 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:16.211390 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:16.211391 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:16.212079 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:16.212080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:16.212642 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:16.212644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:16.213019 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:16.213020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:16.213062 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:16.213588 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:16.213592 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:16.215691 (+ 2099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":109,"threads_started":1}
W20260808 10:08:16.215852 11680 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:16.210814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:53221 (local address 127.2.30.190:38533)
0808 10:08:16.210885 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:16.210887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:16.210944 (+ 57us) server_negotiation.cc:407] Connection header received
0808 10:08:16.211048 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:16.211049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:16.211062 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:16.211128 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:16.211461 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:16.212006 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:16.212734 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:16.212944 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:16.213008 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:16.215785 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:53221: BlockingRecv error: recv got EOF from 127.2.30.131:53221 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 08 10:08:16 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:16 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:16.961607 11684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:16.956868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:38055)
0808 10:08:16.957021 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:16.957031 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:16.957081 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:16.957224 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:16.957225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:16.957397 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:16.957447 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:16.957449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:16.958108 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:16.958108 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:16.958665 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:16.958667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:16.959112 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:16.959114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:16.959179 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:16.959706 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:16.959711 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:16.961509 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
W20260808 10:08:16.961678 11685 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:16.956928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:38055 (local address 127.2.30.190:38533)
0808 10:08:16.957058 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:16.957060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:16.957071 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:08:16.957128 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:16.957129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:16.957139 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:16.957214 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:16.957512 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:16.958037 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:16.958763 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:16.958992 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:16.959097 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:16.961604 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:38055: BlockingRecv error: recv got EOF from 127.2.30.129:38055 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
Aug 08 10:08:17 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:17 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:17.092361 11686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:17.086358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:35643)
0808 10:08:17.086557 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:17.086569 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:17.086639 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:17.086783 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:17.086783 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:17.087030 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:17.087083 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:17.087085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:17.088129 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:17.088131 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:17.088938 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:17.088941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:17.089462 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:17.089464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:17.089532 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:17.090023 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:17.090029 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:17.092277 (+ 2248us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":99,"threads_started":1}
W20260808 10:08:17.092535 11685 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:17.086504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:35643 (local address 127.2.30.190:38533)
0808 10:08:17.086575 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:17.086577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:17.086622 (+ 45us) server_negotiation.cc:407] Connection header received
0808 10:08:17.086700 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:17.086701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:17.086714 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:17.086767 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:17.087193 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:17.088034 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:17.089016 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:17.089353 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:17.089426 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:17.092450 (+ 3024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:35643: BlockingRecv error: recv got EOF from 127.2.30.130:35643 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 10:08:17 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:17 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:17.221498 11687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:17.216377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:57643)
0808 10:08:17.216578 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:17.216590 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:17.216672 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:17.216878 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:17.216879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:17.217060 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:17.217107 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:17.217109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:17.217844 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:17.217845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:17.218371 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:17.218372 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:17.218732 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:17.218734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:17.218799 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:17.219388 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:17.219393 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:17.221432 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":92,"threads_started":1}
W20260808 10:08:17.221658 11685 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:17.216441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:57643 (local address 127.2.30.190:38533)
0808 10:08:17.216535 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:17.216537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:17.216653 (+ 116us) server_negotiation.cc:407] Connection header received
0808 10:08:17.216758 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:17.216759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:17.216778 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:17.216841 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:17.217191 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:17.217742 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:17.218458 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:17.218647 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:17.218721 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:17.221555 (+ 2834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:57643: BlockingRecv error: recv got EOF from 127.2.30.131:57643 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 10:08:17 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:17 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:17.967495 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:17.962160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:55575)
0808 10:08:17.962420 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:17.962432 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:17.962507 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:17.962655 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:17.962655 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:17.962839 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:17.962888 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:17.962890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:17.963617 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:17.963618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:17.964199 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:17.964202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:17.964565 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:17.964565 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:17.964605 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:17.965146 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:17.965150 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:17.967430 (+ 2280us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":176,"threads_started":1}
W20260808 10:08:17.967646 11689 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:17.962318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:55575 (local address 127.2.30.190:38533)
0808 10:08:17.962438 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:17.962440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:17.962508 (+ 68us) server_negotiation.cc:407] Connection header received
0808 10:08:17.962558 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:17.962559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:17.962570 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:17.962634 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:17.962957 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:17.963532 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:17.964295 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:17.964494 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:17.964550 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:17.967554 (+ 3004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:55575: BlockingRecv error: recv got EOF from 127.2.30.129:55575 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
Aug 08 10:08:18 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:18 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:18.098488 11690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:18.092908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:47619)
0808 10:08:18.093086 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:18.093099 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:18.093181 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:18.093370 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:18.093371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:18.093601 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:18.093679 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:18.093681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:18.094732 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:18.094735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:18.095337 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:18.095339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:18.095724 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:18.095725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:18.095784 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:18.096287 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:18.096292 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:18.098388 (+ 2096us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
W20260808 10:08:18.098577 11689 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:18.093096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:47619 (local address 127.2.30.190:38533)
0808 10:08:18.093170 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:18.093172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:18.093190 (+ 18us) server_negotiation.cc:407] Connection header received
0808 10:08:18.093264 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:18.093265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:18.093277 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:18.093336 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:18.093770 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:18.094639 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:18.095439 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:18.095650 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:18.095726 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:18.098492 (+ 2766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:47619: BlockingRecv error: recv got EOF from 127.2.30.130:47619 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 08 10:08:18 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:18 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:18.227867 11691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:18.222103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:46789)
0808 10:08:18.222325 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:18.222335 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:18.222398 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:18.222557 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:18.222558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:18.222740 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:18.222788 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:18.222790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:18.223514 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:18.223515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:18.224127 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:18.224129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:18.224570 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:18.224571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:18.224624 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:18.225090 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:18.225095 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:18.227798 (+ 2703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":97,"threads_started":1}
W20260808 10:08:18.228022 11689 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:18.222187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:46789 (local address 127.2.30.190:38533)
0808 10:08:18.222271 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:18.222274 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:18.222423 (+ 149us) server_negotiation.cc:407] Connection header received
0808 10:08:18.222459 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:18.222460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:18.222474 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:18.222540 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:18.222853 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:18.223425 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:18.224277 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:18.224468 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:18.224532 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:18.227933 (+ 3401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:46789: BlockingRecv error: recv got EOF from 127.2.30.131:46789 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 10:08:18 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:18 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:18.973793 11692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:18.968170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:49437)
0808 10:08:18.968424 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:18.968438 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:18.968518 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:18.968726 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:18.968728 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:18.968999 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:18.969062 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:18.969065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:18.969803 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:18.969804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:18.970400 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:18.970402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:18.970843 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:18.970844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:18.970890 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:18.971496 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:18.971501 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:18.973660 (+ 2159us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":145,"threads_started":1}
W20260808 10:08:18.973843 11693 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:18.968262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:49437 (local address 127.2.30.190:38533)
0808 10:08:18.968411 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:18.968414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:18.968515 (+ 101us) server_negotiation.cc:407] Connection header received
0808 10:08:18.968597 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:18.968597 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:18.968610 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:18.968694 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:18.969164 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:18.969703 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:18.970512 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:18.970718 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:18.970829 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:18.973765 (+ 2936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:49437: BlockingRecv error: recv got EOF from 127.2.30.129:49437 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":60,"threads_started":1}
Aug 08 10:08:19 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:19 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:19.104619 11694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:19.099119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:35179)
0808 10:08:19.099346 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:19.099356 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:19.099449 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:19.099634 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:19.099636 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:19.099871 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:19.099938 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:19.099941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:19.100953 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:19.100954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:19.101564 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:19.101568 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:19.101972 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:19.101973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:19.102023 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:19.102501 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:19.102507 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:19.104512 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":79,"threads_started":1}
W20260808 10:08:19.104671 11693 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:19.099184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:35179 (local address 127.2.30.190:38533)
0808 10:08:19.099277 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:19.099279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:19.099432 (+ 153us) server_negotiation.cc:407] Connection header received
0808 10:08:19.099523 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:19.099524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:19.099539 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:19.099628 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:19.100036 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:19.100850 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:19.101664 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:19.101867 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:19.101933 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:19.104601 (+ 2668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:35179: BlockingRecv error: recv got EOF from 127.2.30.130:35179 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 10:08:19 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:19 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:19.234020 11695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:19.228487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:50163)
0808 10:08:19.228637 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:19.228655 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:19.228712 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:19.228928 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:19.228929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:19.229115 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:19.229169 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:19.229170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:19.229973 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:19.229974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:19.230595 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:19.230598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:19.231028 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:19.231030 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:19.231109 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:19.231734 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:19.231739 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:19.233919 (+ 2180us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
W20260808 10:08:19.234136 11693 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:19.228611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:50163 (local address 127.2.30.190:38533)
0808 10:08:19.228679 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:19.228681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:19.228695 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:08:19.228817 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:19.228818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:19.228834 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:19.228910 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:19.229277 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:19.229867 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:19.230687 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:19.230881 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:19.230940 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:19.234049 (+ 3109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:50163: BlockingRecv error: recv got EOF from 127.2.30.131:50163 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 10:08:19 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:19 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:19.984691 11696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:19.974421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:52745)
0808 10:08:19.974622 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:19.974634 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:19.974711 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:19.974868 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:19.974869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:19.975066 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:19.975118 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:19.975120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:19.975828 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:19.975829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:19.976475 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:19.976478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:19.976833 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:19.976833 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:19.976885 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:19.982605 (+ 5720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:19.982611 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:19.984582 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":99,"threads_started":1}
W20260808 10:08:19.984767 11697 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:19.974574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:52745 (local address 127.2.30.190:38533)
0808 10:08:19.974706 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:19.974708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:19.974717 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:08:19.974769 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:19.974770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:19.974781 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:19.974856 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:19.975182 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:19.975729 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:19.976561 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:19.976767 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:19.976824 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:19.984706 (+ 7882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:52745: BlockingRecv error: recv got EOF from 127.2.30.129:52745 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
Aug 08 10:08:20 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:20 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:20.110430 11698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:20.105160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:53205)
0808 10:08:20.105381 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:20.105392 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:20.105486 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:20.105630 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:20.105631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:20.105825 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:20.105882 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:20.105884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:20.106747 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:20.106749 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:20.107353 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:20.107356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:20.107756 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:20.107757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:20.107834 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:20.108354 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:20.108360 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:20.110326 (+ 1966us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":111,"threads_started":1}
W20260808 10:08:20.110486 11697 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:20.105252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:53205 (local address 127.2.30.190:38533)
0808 10:08:20.105323 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:20.105325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:20.105465 (+ 140us) server_negotiation.cc:407] Connection header received
0808 10:08:20.105559 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:20.105560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:20.105573 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:20.105626 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:20.105979 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:20.106645 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:20.107452 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:20.107662 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:20.107753 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:20.110417 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:53205: BlockingRecv error: recv got EOF from 127.2.30.130:53205 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 10:08:20 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:20 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:20.241340 11699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:20.234773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:51361)
0808 10:08:20.235061 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:20.235072 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:20.235162 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:20.235365 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:20.235366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:20.235605 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:20.235684 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:20.235687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:20.236738 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:20.236739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:20.237773 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:20.237777 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:20.238265 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:20.238266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:20.238333 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:20.238948 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:20.238954 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:20.241248 (+ 2294us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":77,"threads_started":1}
W20260808 10:08:20.241449 11697 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:20.234846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:51361 (local address 127.2.30.190:38533)
0808 10:08:20.234917 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:20.234920 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:20.235170 (+ 250us) server_negotiation.cc:407] Connection header received
0808 10:08:20.235220 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:20.235221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:20.235246 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:20.235339 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:20.235805 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:20.236655 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:20.237883 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:20.238187 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:20.238262 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:20.241382 (+ 3120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:51361: BlockingRecv error: recv got EOF from 127.2.30.131:51361 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 08 10:08:20 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:20 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:20.990417 11700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:20.985235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:42423)
0808 10:08:20.985412 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:20.985423 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:20.985476 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:20.985664 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:20.985665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:20.985849 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:20.985896 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:20.985897 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:20.986579 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:20.986580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:20.987283 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:20.987286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:20.987759 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:20.987760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:20.987822 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:20.988317 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:20.988322 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:20.990326 (+ 2004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":120,"threads_started":1}
W20260808 10:08:20.990521 11701 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:20.985355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:42423 (local address 127.2.30.190:38533)
0808 10:08:20.985508 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:20.985510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:20.985534 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:08:20.985561 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:20.985562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:20.985573 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:20.985638 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:20.985968 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:20.986498 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:20.987389 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:20.987624 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:20.987734 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:20.990445 (+ 2711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:42423: BlockingRecv error: recv got EOF from 127.2.30.129:42423 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
Aug 08 10:08:21 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:21 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:21.115893 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:21.110939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:41745)
0808 10:08:21.111129 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:21.111141 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:21.111214 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:21.111388 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:21.111389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:21.111563 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:21.111615 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:21.111617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:21.112299 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:21.112299 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:21.112847 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:21.112848 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:21.113233 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:21.113234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:21.113276 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:21.113860 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:21.113864 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:21.115827 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":100,"threads_started":1}
W20260808 10:08:21.116080 11701 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:21.111004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:41745 (local address 127.2.30.190:38533)
0808 10:08:21.111064 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:21.111066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:21.111270 (+ 204us) server_negotiation.cc:407] Connection header received
0808 10:08:21.111306 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:21.111307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:21.111320 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:21.111372 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:21.111710 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:21.112209 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:21.112939 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:21.113137 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:21.113228 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:21.115989 (+ 2761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:41745: BlockingRecv error: recv got EOF from 127.2.30.130:41745 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 10:08:21 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:21 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:21.246858 11703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:21.241910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:44521)
0808 10:08:21.242149 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:21.242161 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:21.242252 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:21.242408 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:21.242409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:21.242604 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:21.242650 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:21.242652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:21.243393 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:21.243394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:21.243930 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:21.243932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:21.244287 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:21.244288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:21.244362 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:21.244861 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:21.244866 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:21.246753 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":133,"threads_started":1}
W20260808 10:08:21.246928 11701 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:21.242046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:44521 (local address 127.2.30.190:38533)
0808 10:08:21.242128 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:21.242130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:21.242232 (+ 102us) server_negotiation.cc:407] Connection header received
0808 10:08:21.242311 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:21.242312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:21.242327 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:21.242382 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:21.242732 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:21.243306 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:21.244014 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:21.244204 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:21.244275 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:21.246857 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:44521: BlockingRecv error: recv got EOF from 127.2.30.131:44521 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 10:08:21 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:21 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:21.996250 11704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:21.991003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:47313)
0808 10:08:21.991152 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:21.991166 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:21.991225 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:21.991437 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:21.991438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:21.991647 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:21.991696 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:21.991699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:21.992453 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:21.992454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:21.993031 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:21.993033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:21.993659 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:21.993660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:21.993717 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:21.994259 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:21.994264 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:21.996187 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
W20260808 10:08:21.996398 11705 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:21.991062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:47313 (local address 127.2.30.190:38533)
0808 10:08:21.991226 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:21.991228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:21.991291 (+ 63us) server_negotiation.cc:407] Connection header received
0808 10:08:21.991320 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:21.991320 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:21.991335 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:21.991416 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:21.991778 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:21.992367 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:21.993114 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:21.993391 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:21.993616 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:21.996340 (+ 2724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:47313: BlockingRecv error: recv got EOF from 127.2.30.129:47313 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
Aug 08 10:08:22 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:22 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:22.121487 11706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:22.116436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:48711)
0808 10:08:22.116587 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:22.116605 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:22.116675 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:22.116850 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:22.116851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:22.117028 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:22.117079 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:22.117081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:22.117872 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:22.117873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:22.118425 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:22.118428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:22.118785 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:22.118786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:22.118829 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:22.119417 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:22.119422 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:22.121412 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
W20260808 10:08:22.121624 11705 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:22.116535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:48711 (local address 127.2.30.190:38533)
0808 10:08:22.116626 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:22.116628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:22.116658 (+ 30us) server_negotiation.cc:407] Connection header received
0808 10:08:22.116745 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:22.116746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:22.116759 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:22.116820 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:22.117182 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:22.117789 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:22.118513 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:22.118713 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:22.118773 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:22.121536 (+ 2763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:48711: BlockingRecv error: recv got EOF from 127.2.30.130:48711 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 10:08:22 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:22 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:22.252223 11707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:22.247367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:49043)
0808 10:08:22.247520 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:22.247531 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:22.247592 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:22.247722 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:22.247723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:22.247891 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:22.247946 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:22.247947 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:22.248591 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:22.248593 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:22.249113 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:22.249115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:22.249471 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:22.249472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:22.249517 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:22.250051 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:22.250056 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:22.252133 (+ 2077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":59,"threads_started":1}
W20260808 10:08:22.252396 11705 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:22.247497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:49043 (local address 127.2.30.190:38533)
0808 10:08:22.247543 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:22.247546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:22.247575 (+ 29us) server_negotiation.cc:407] Connection header received
0808 10:08:22.247637 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:22.247637 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:22.247649 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:22.247697 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:22.248007 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:22.248515 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:22.249192 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:22.249394 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:22.249452 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:22.252294 (+ 2842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:49043: BlockingRecv error: recv got EOF from 127.2.30.131:49043 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 08 10:08:22 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:22 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:23.003038 11708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:22.997009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:35505)
0808 10:08:22.997199 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:22.997213 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:22.997283 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:22.997422 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:22.997423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:22.997607 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:22.997657 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:22.997658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:22.998478 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:22.998479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:22.999137 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:22.999140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:22.999578 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:22.999579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:22.999628 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:23.000212 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:23.000217 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:23.002901 (+ 2684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":120,"threads_started":1}
W20260808 10:08:23.003242 11709 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:22.997147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:35505 (local address 127.2.30.190:38533)
0808 10:08:22.997281 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:22.997283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:22.997295 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:08:22.997328 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:22.997329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:22.997340 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:22.997406 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:22.997759 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:22.998375 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:22.999228 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:22.999497 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:22.999564 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:23.003074 (+ 3510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:35505: BlockingRecv error: recv got EOF from 127.2.30.129:35505 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
Aug 08 10:08:23 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:23 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:23.127538 11710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:23.122105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:60857)
0808 10:08:23.122272 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:23.122284 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:23.122358 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:23.122524 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:23.122525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:23.122720 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:23.122787 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:23.122789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:23.123612 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:23.123614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:23.124299 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:23.124302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:23.124704 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:23.124705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:23.124770 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:23.125301 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:23.125306 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:23.127419 (+ 2113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
W20260808 10:08:23.127595 11709 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:23.122232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:60857 (local address 127.2.30.190:38533)
0808 10:08:23.122293 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:23.122295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:23.122362 (+ 67us) server_negotiation.cc:407] Connection header received
0808 10:08:23.122412 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:23.122412 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:23.122425 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:23.122495 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:23.122885 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:23.123498 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:23.124400 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:23.124603 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:23.124690 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:23.127525 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:60857: BlockingRecv error: recv got EOF from 127.2.30.130:60857 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 10:08:23 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:23 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:23.258665 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:23.252842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:46341)
0808 10:08:23.253111 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:23.253131 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:23.253201 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:23.253403 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:23.253405 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:23.253587 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:23.253639 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:23.253641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:23.254339 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:23.254341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:23.254922 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:23.254924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:23.255338 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:23.255339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:23.255513 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:23.256100 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:23.256105 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:23.258548 (+ 2443us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":203,"threads_started":1}
W20260808 10:08:23.258715 11709 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:23.252951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:46341 (local address 127.2.30.190:38533)
0808 10:08:23.253004 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:23.253007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:23.253199 (+ 192us) server_negotiation.cc:407] Connection header received
0808 10:08:23.253257 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:23.253258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:23.253272 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:23.253385 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:23.253757 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:23.254257 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:23.255041 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:23.255258 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:23.255329 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:23.258661 (+ 3332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:46341: BlockingRecv error: recv got EOF from 127.2.30.131:46341 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 10:08:24 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:24 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:24.008754 11712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:24.003516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:42049)
0808 10:08:24.003675 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:24.003687 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:24.003742 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:24.003895 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:24.003895 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:24.004078 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:24.004130 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:24.004132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:24.004847 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:24.004848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:24.005374 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:24.005376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:24.005890 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:24.005891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:24.005942 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:24.006448 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:24.006453 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:24.008625 (+ 2172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":96,"threads_started":1}
W20260808 10:08:24.008821 11713 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:24.003615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:42049 (local address 127.2.30.190:38533)
0808 10:08:24.003728 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:24.003730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:24.003741 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:08:24.003799 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:24.003800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:24.003811 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:24.003861 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:24.004199 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:24.004750 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:24.005487 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:24.005743 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:24.005801 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:24.008728 (+ 2927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:42049: BlockingRecv error: recv got EOF from 127.2.30.129:42049 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":54,"threads_started":1}
Aug 08 10:08:24 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:24 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:24.133741 11714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:24.128342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:60373)
0808 10:08:24.128662 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:24.128676 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:24.128750 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:24.128895 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:24.128896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:24.129080 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:24.129127 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:24.129129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:24.129819 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:24.129820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:24.130464 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:24.130467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:24.130875 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:24.130877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:24.130932 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:24.131549 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:24.131554 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:24.133625 (+ 2071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":234,"threads_started":1}
W20260808 10:08:24.133803 11713 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:24.128430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:60373 (local address 127.2.30.190:38533)
0808 10:08:24.128548 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:24.128550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:24.128789 (+ 239us) server_negotiation.cc:407] Connection header received
0808 10:08:24.128821 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:24.128821 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:24.128833 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:24.128892 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:24.129235 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:24.129725 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:24.130569 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:24.130774 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:24.130843 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:24.133734 (+ 2891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:60373: BlockingRecv error: recv got EOF from 127.2.30.130:60373 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 10:08:24 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:24 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:24.264828 11715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:24.259399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:46967)
0808 10:08:24.259596 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:24.259607 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:24.259681 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:24.259847 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:24.259848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:24.260044 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:24.260093 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:24.260095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:24.260929 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:24.260930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:24.261824 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:24.261827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:24.262208 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:24.262209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:24.262270 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:24.262822 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:24.262828 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:24.264701 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":125,"threads_started":1}
W20260808 10:08:24.264860 11713 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:24.259491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:46967 (local address 127.2.30.190:38533)
0808 10:08:24.259568 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:24.259571 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:24.259683 (+ 112us) server_negotiation.cc:407] Connection header received
0808 10:08:24.259733 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:24.259734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:24.259748 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:24.259809 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:24.260203 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:24.260783 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:24.261913 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:24.262114 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:24.262196 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:24.264789 (+ 2593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:46967: BlockingRecv error: recv got EOF from 127.2.30.131:46967 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 10:08:25 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:25 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:25.019645 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:25.009431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:42113)
0808 10:08:25.009584 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:25.009604 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:25.009692 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:25.009930 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:25.009931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:25.010120 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:25.010171 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:25.010173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:25.011048 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:25.011049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:25.011672 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:25.011676 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:25.012053 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:25.012053 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:25.012098 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:25.017431 (+ 5333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:25.017437 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:25.019582 (+ 2145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
W20260808 10:08:25.019820 11717 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:25.009507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:42113 (local address 127.2.30.190:38533)
0808 10:08:25.009667 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:25.009669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:25.009694 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:08:25.009761 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:25.009762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:25.009775 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:25.009863 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:25.010298 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:25.010916 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:25.011772 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:25.011978 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:25.012035 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:25.019718 (+ 7683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:42113: BlockingRecv error: recv got EOF from 127.2.30.129:42113 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
Aug 08 10:08:25 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:25 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:25.139963 11718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:25.134431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:44935)
0808 10:08:25.134596 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:25.134609 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:25.134694 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:25.134839 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:25.134840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:25.135112 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:25.135255 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:25.135258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:25.136101 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:25.136103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:25.136663 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:25.136666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:25.137078 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:25.137080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:25.137147 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:25.137669 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:25.137674 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:25.139815 (+ 2141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":78,"threads_started":1}
W20260808 10:08:25.140008 11717 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:25.134542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:44935 (local address 127.2.30.190:38533)
0808 10:08:25.134597 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:25.134599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:25.134711 (+ 112us) server_negotiation.cc:407] Connection header received
0808 10:08:25.134749 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:25.134750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:25.134764 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:25.134822 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:25.135421 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:25.135981 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:25.136775 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:25.136981 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:25.137074 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:25.139927 (+ 2853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:44935: BlockingRecv error: recv got EOF from 127.2.30.130:44935 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 08 10:08:25 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:25 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:25.270699 11719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:25.265424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:35869)
0808 10:08:25.265653 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:25.265665 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:25.265763 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:25.265950 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:25.265951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:25.266151 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:25.266199 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:25.266201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:25.266907 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:25.266908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:25.267523 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:25.267525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:25.267902 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:25.267903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:25.267965 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:25.268496 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:25.268501 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:25.270610 (+ 2109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":121,"threads_started":1}
W20260808 10:08:25.270850 11717 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:25.265531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:35869 (local address 127.2.30.190:38533)
0808 10:08:25.265628 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:25.265630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:25.265755 (+ 125us) server_negotiation.cc:407] Connection header received
0808 10:08:25.265828 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:25.265829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:25.265844 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:25.265925 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:25.266299 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:25.266814 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:25.267620 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:25.267823 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:25.267903 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:25.270743 (+ 2840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:35869: BlockingRecv error: recv got EOF from 127.2.30.131:35869 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 08 10:08:26 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:26 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:26.025583 11720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:26.020335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:58797)
0808 10:08:26.020499 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:26.020513 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:26.020600 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:26.020810 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:26.020811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:26.021007 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:26.021083 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:26.021085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:26.021848 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:26.021849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:26.022411 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:26.022414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:26.022813 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:26.022814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:26.022878 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:26.023483 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:26.023488 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:26.025452 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":77,"threads_started":1}
W20260808 10:08:26.025852 11721 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:26.020397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:58797 (local address 127.2.30.190:38533)
0808 10:08:26.020583 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:26.020585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:26.020595 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:08:26.020692 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:26.020693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:26.020707 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:26.020778 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:26.021163 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:26.021734 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:26.022536 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:26.022738 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:26.022801 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:26.025774 (+ 2973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:58797: BlockingRecv error: recv got EOF from 127.2.30.129:58797 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":87,"threads_started":1}
Aug 08 10:08:26 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:26 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:26.146278 11722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:26.140611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:41743)
0808 10:08:26.140809 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:26.140821 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:26.140902 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:26.141097 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:26.141099 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:26.141471 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:26.141537 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:26.141541 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:26.142524 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:26.142525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:26.143176 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:26.143178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:26.143598 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:26.143599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:26.143649 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:26.144333 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:26.144337 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:26.146216 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":98,"threads_started":1}
W20260808 10:08:26.146492 11721 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:26.140771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:41743 (local address 127.2.30.190:38533)
0808 10:08:26.140820 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:26.140822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:26.140881 (+ 59us) server_negotiation.cc:407] Connection header received
0808 10:08:26.140960 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:26.140961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:26.140974 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:26.141033 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:26.141712 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:26.142426 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:26.143305 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:26.143508 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:26.143570 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:26.146325 (+ 2755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:41743: BlockingRecv error: recv got EOF from 127.2.30.130:41743 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 08 10:08:26 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:26 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.131@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:26.277359 11723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:26.271287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.131:53853)
0808 10:08:26.271549 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:26.271560 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:26.271656 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:26.271847 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:26.271849 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:26.272148 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:26.272223 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:26.272226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:26.273302 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:26.273303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:26.274222 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:26.274225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:26.274582 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:26.274583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:26.274631 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:26.275321 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:26.275327 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:26.277262 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":171,"threads_started":1}
W20260808 10:08:26.277509 11721 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:26.271409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:53853 (local address 127.2.30.190:38533)
0808 10:08:26.271467 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:26.271469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:26.271634 (+ 165us) server_negotiation.cc:407] Connection header received
0808 10:08:26.271737 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:26.271739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:26.271758 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:26.271827 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:26.272419 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:26.273181 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:26.274298 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:26.274495 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:26.274560 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:26.277392 (+ 2832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.131:53853: BlockingRecv error: recv got EOF from 127.2.30.131:53853 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 08 10:08:27 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:27 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.129@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:27.031466 11724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:27.026265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.129:42045)
0808 10:08:27.026420 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:27.026431 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:27.026491 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:27.026649 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:27.026649 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:27.026880 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:27.026964 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:27.026966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.027658 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.027659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:27.028207 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:27.028209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.028553 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.028554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:27.028594 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:27.029122 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:27.029127 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:27.031370 (+ 2243us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
W20260808 10:08:27.031612 11725 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:27.026374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:42045 (local address 127.2.30.190:38533)
0808 10:08:27.026505 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:27.026507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:27.026518 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:08:27.026552 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:27.026553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:27.026564 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:27.026624 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:27.027077 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.027584 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.028285 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.028485 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.028541 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:27.031515 (+ 2974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.129:42045: BlockingRecv error: recv got EOF from 127.2.30.129:42045 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
Aug 08 10:08:27 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 10:08:27 dist-test-slave-w206 krb5kdc[10625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.30.130@KRBTEST.COM for oryx/127.2.30.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 10:08:27.152374 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:27.147044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38533 (local address 127.2.30.130:43805)
0808 10:08:27.147202 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:27.147216 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:27.147309 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:27.147507 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:27.147509 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:27.147710 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:27.147768 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:27.147770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.148431 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.148432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:27.149009 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:27.149011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.149392 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.149394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:27.149450 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:27.150077 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:27.150082 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:27.152267 (+ 2185us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.30.190:38533: Server oryx/127.2.30.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
W20260808 10:08:27.152513 11725 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:27.147137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:43805 (local address 127.2.30.190:38533)
0808 10:08:27.147208 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:27.147210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:27.147291 (+ 81us) server_negotiation.cc:407] Connection header received
0808 10:08:27.147372 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:27.147373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:27.147384 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:27.147455 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:27.147851 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.148344 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.149090 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.149299 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.149364 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:27.152442 (+ 3078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.30.130:43805: BlockingRecv error: recv got EOF from 127.2.30.130:43805 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:08:27.185834 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 11231
I20260808 10:08:27.192283 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 11391
I20260808 10:08:27.198697 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 11528
I20260808 10:08:27.204185 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 11158
2026-08-08T10:08:27Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17633 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T10:08:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:08:27Z Disabled control of system clock
I20260808 10:08:27.236092 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38879
--webserver_interface=127.2.30.190
--webserver_port=0
--builtin_ntp_servers=127.2.30.148:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:38879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 10:08:27.322556 11730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:27.322798 11730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:27.322829 11730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:27.324108 11730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:08:27.324156 11730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:27.324169 11730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:27.324182 11730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:08:27.324194 11730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:08:27.325536 11730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:38879
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:27.325744 11730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:27.326083 11730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:27.331677 11736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:27.331694 11735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:27.331732 11730 server_base.cc:1034] running on GCE node
W20260808 10:08:27.331700 11738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:27.332217 11730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:27.332509 11730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:27.333662 11730 hybrid_clock.cc:648] HybridClock initialized: now 1786183707333631 us; error 45 us; skew 500 ppm
I20260808 10:08:27.335108 11730 webserver.cc:460] Webserver started at http://127.2.30.190:46617/ using document root <none> and password file <none>
I20260808 10:08:27.335371 11730 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:27.335441 11730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:27.335599 11730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:27.336750 11730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "d4b7d6c2f8d04a7688633fd1b605ebf9"
format_stamp: "Formatted at 2026-08-08 10:08:27 on dist-test-slave-w206"
server_key: "52025e25aee0a42ff7bb729e9f23ba98"
server_key_iv: "c832430d62b09c7657822347cc823fa9"
server_key_version: "encryptionkey@0"
I20260808 10:08:27.337141 11730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "d4b7d6c2f8d04a7688633fd1b605ebf9"
format_stamp: "Formatted at 2026-08-08 10:08:27 on dist-test-slave-w206"
server_key: "52025e25aee0a42ff7bb729e9f23ba98"
server_key_iv: "c832430d62b09c7657822347cc823fa9"
server_key_version: "encryptionkey@0"
I20260808 10:08:27.338562 11730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:08:27.339465 11744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:27.339659 11730 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:08:27.339773 11730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "d4b7d6c2f8d04a7688633fd1b605ebf9"
format_stamp: "Formatted at 2026-08-08 10:08:27 on dist-test-slave-w206"
server_key: "52025e25aee0a42ff7bb729e9f23ba98"
server_key_iv: "c832430d62b09c7657822347cc823fa9"
server_key_version: "encryptionkey@0"
I20260808 10:08:27.339861 11730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:27.357133 11730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:27.357488 11730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:27.357663 11730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:27.362207 11730 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:38879
I20260808 10:08:27.362385 11801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:38879 every 8 connection(s)
I20260808 10:08:27.362722 11730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:08:27.363648 11802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:27.366436 11802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9: Bootstrap starting.
I20260808 10:08:27.367187 11802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:27.367511 11802 log.cc:824] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:27.368248 11802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9: No bootstrap required, opened a new log
I20260808 10:08:27.369916 11802 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b7d6c2f8d04a7688633fd1b605ebf9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38879 } }
I20260808 10:08:27.370067 11802 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:27.370090 11802 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4b7d6c2f8d04a7688633fd1b605ebf9, State: Initialized, Role: FOLLOWER
I20260808 10:08:27.370265 11802 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b7d6c2f8d04a7688633fd1b605ebf9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38879 } }
I20260808 10:08:27.370347 11802 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:08:27.370393 11802 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:08:27.370441 11802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:27.371104 11802 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b7d6c2f8d04a7688633fd1b605ebf9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38879 } }
I20260808 10:08:27.371197 11802 leader_election.cc:302] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4b7d6c2f8d04a7688633fd1b605ebf9; no voters:
I20260808 10:08:27.371404 11802 leader_election.cc:288] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:08:27.371510 11805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:08:27.371791 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 11730
I20260808 10:08:27.371882 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:08:27.372079 2170 external_mini_cluster.cc:1428] Setting key 7828740f84ca8e05dd9158b4b50990b2
I20260808 10:08:27.371713 11802 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:08:27.372383 11805 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [term 1 LEADER]: Becoming Leader. State: Replica: d4b7d6c2f8d04a7688633fd1b605ebf9, State: Running, Role: LEADER
I20260808 10:08:27.372546 11805 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b7d6c2f8d04a7688633fd1b605ebf9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38879 } }
I20260808 10:08:27.373512 11806 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4b7d6c2f8d04a7688633fd1b605ebf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b7d6c2f8d04a7688633fd1b605ebf9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38879 } } }
I20260808 10:08:27.373618 11806 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:27.373971 11807 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4b7d6c2f8d04a7688633fd1b605ebf9. Latest consensus state: current_term: 1 leader_uuid: "d4b7d6c2f8d04a7688633fd1b605ebf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4b7d6c2f8d04a7688633fd1b605ebf9" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38879 } } }
I20260808 10:08:27.374060 11807 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9 [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:27.374184 11816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:08:27.374783 11816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:08:27.376596 11816 catalog_manager.cc:1347] Generated new cluster ID: 2e781a7fcaee45698c2206f61919d4f2
I20260808 10:08:27.376653 11816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:08:27.378793 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:27.372653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34650 (local address 127.2.30.190:38879)
0808 10:08:27.373372 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:27.373381 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:27.373417 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:08:27.373537 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:27.373545 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:27.373621 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:27.374047 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:27.374152 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.375605 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.376537 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.377126 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.377241 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:27.378041 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:27.378045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:27.378046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:08:27.378253 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:27.378296 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:08:27.378310 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:27.378371 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:27.378480 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 10:08:27.378562 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":115,"threads_started":1}
I20260808 10:08:27.383524 11816 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:08:27.384272 11816 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:08:27.392490 11816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4b7d6c2f8d04a7688633fd1b605ebf9: Generated new TSK 0
I20260808 10:08:27.392897 11816 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:08:27.399303 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38879
--builtin_ntp_servers=127.2.30.148:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 10:08:27.485926 11826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:27.486191 11826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:27.486231 11826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:27.487567 11826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:27.487633 11826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:27.487679 11826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:08:27.489095 11826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:27.489400 11826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:27.489733 11826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:27.496105 11831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:27.496112 11834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:27.496112 11832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:27.496244 11826 server_base.cc:1034] running on GCE node
I20260808 10:08:27.496862 11826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:27.497124 11826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:27.498325 11826 hybrid_clock.cc:648] HybridClock initialized: now 1786183707498303 us; error 43 us; skew 500 ppm
I20260808 10:08:27.499944 11826 webserver.cc:460] Webserver started at http://127.2.30.129:45069/ using document root <none> and password file <none>
I20260808 10:08:27.500208 11826 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:27.500281 11826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:27.500433 11826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:27.501577 11826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "61d057f83135497e98feb0f27214f319"
format_stamp: "Formatted at 2026-08-08 10:08:27 on dist-test-slave-w206"
server_key: "0bccf4e8880649ad2a158021855a2af8"
server_key_iv: "5b68fde57965314fe12a2f97928371a0"
server_key_version: "encryptionkey@0"
I20260808 10:08:27.501974 11826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "61d057f83135497e98feb0f27214f319"
format_stamp: "Formatted at 2026-08-08 10:08:27 on dist-test-slave-w206"
server_key: "0bccf4e8880649ad2a158021855a2af8"
server_key_iv: "5b68fde57965314fe12a2f97928371a0"
server_key_version: "encryptionkey@0"
I20260808 10:08:27.503515 11826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:08:27.504621 11842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:27.504917 11826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:08:27.505034 11826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "61d057f83135497e98feb0f27214f319"
format_stamp: "Formatted at 2026-08-08 10:08:27 on dist-test-slave-w206"
server_key: "0bccf4e8880649ad2a158021855a2af8"
server_key_iv: "5b68fde57965314fe12a2f97928371a0"
server_key_version: "encryptionkey@0"
I20260808 10:08:27.505115 11826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:27.523545 11826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:27.523905 11826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:27.524034 11826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:27.524283 11826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:27.524624 11826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:27.524660 11826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:27.524695 11826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:27.524715 11826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:27.531484 11826 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:37415
I20260808 10:08:27.531613 11959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:37415 every 8 connection(s)
I20260808 10:08:27.532011 11826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:08:27.536043 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 11826
I20260808 10:08:27.536336 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:08:27.536595 2170 external_mini_cluster.cc:1428] Setting key 21e6dec2a22c6387003faa0baf7000d2
I20260808 10:08:27.537750 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38879
--builtin_ntp_servers=127.2.30.148:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 10:08:27.538281 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:27.533012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:35503 (local address 127.2.30.190:38879)
0808 10:08:27.533162 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:27.533165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:27.533390 (+ 225us) server_negotiation.cc:407] Connection header received
0808 10:08:27.534010 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:27.534012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:27.534028 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:27.534084 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:27.534825 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.535457 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.536594 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.536940 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.537004 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:27.537975 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:27.537977 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:27.537978 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:08:27.537982 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:27.537999 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:08:27.538005 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:27.538036 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:27.538156 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 10:08:27.538198 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 10:08:27.538529 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:27.532919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38879 (local address 127.2.30.129:35503)
0808 10:08:27.533254 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:27.533292 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:27.533892 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:27.534166 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:27.534170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:27.534179 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:27.534707 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:27.534712 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.535561 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.535562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:27.536486 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:27.536489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.537013 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.537014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:27.537165 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:27.537792 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:08:27.537792 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:27.537822 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:08:27.537865 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:27.538071 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:27.538077 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:27.538079 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:08:27.538157 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 10:08:27.538274 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":121,"threads_started":1}
I20260808 10:08:27.539179 11960 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38879
I20260808 10:08:27.539304 11960 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:27.539539 11960 heartbeater.cc:511] Master 127.2.30.190:38879 requested a full tablet report, sending...
I20260808 10:08:27.540113 11762 ts_manager.cc:194] Registered new tserver with Master: 61d057f83135497e98feb0f27214f319 (127.2.30.129:37415)
I20260808 10:08:27.540789 11762 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.30.129:35503
W20260808 10:08:27.623349 11963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:27.623589 11963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:27.623613 11963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:27.624883 11963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:27.624935 11963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:27.624962 11963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:08:27.626438 11963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:27.626720 11963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:27.627111 11963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:27.632557 11969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:27.632581 11968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:27.632557 11971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:27.633143 11963 server_base.cc:1034] running on GCE node
I20260808 10:08:27.633412 11963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:27.633704 11963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:27.634891 11963 hybrid_clock.cc:648] HybridClock initialized: now 1786183707634887 us; error 53 us; skew 500 ppm
I20260808 10:08:27.636389 11963 webserver.cc:460] Webserver started at http://127.2.30.130:41179/ using document root <none> and password file <none>
I20260808 10:08:27.636669 11963 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:27.636775 11963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:27.636932 11963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:27.638146 11963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "34430bb0413e4191867cd8bf91732c21"
format_stamp: "Formatted at 2026-08-08 10:08:27 on dist-test-slave-w206"
server_key: "b8b2b4c48e25b3b9ce593d347eb1182b"
server_key_iv: "fe1ed920164e131e9b19c8bd2edb31e6"
server_key_version: "encryptionkey@0"
I20260808 10:08:27.638557 11963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "34430bb0413e4191867cd8bf91732c21"
format_stamp: "Formatted at 2026-08-08 10:08:27 on dist-test-slave-w206"
server_key: "b8b2b4c48e25b3b9ce593d347eb1182b"
server_key_iv: "fe1ed920164e131e9b19c8bd2edb31e6"
server_key_version: "encryptionkey@0"
I20260808 10:08:27.640209 11963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 10:08:27.641081 11977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:27.641278 11963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:27.641412 11963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "34430bb0413e4191867cd8bf91732c21"
format_stamp: "Formatted at 2026-08-08 10:08:27 on dist-test-slave-w206"
server_key: "b8b2b4c48e25b3b9ce593d347eb1182b"
server_key_iv: "fe1ed920164e131e9b19c8bd2edb31e6"
server_key_version: "encryptionkey@0"
I20260808 10:08:27.641496 11963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:27.660107 11963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:27.660492 11963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:27.660677 11963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:27.660941 11963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:27.661325 11963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:27.661384 11963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:27.661427 11963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:27.661470 11963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:27.667873 11963 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:37445
I20260808 10:08:27.667974 12096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:37445 every 8 connection(s)
I20260808 10:08:27.668334 11963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:08:27.674652 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 11963
I20260808 10:08:27.674788 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:08:27.675060 2170 external_mini_cluster.cc:1428] Setting key 92989eeea40f9993e473171e549b3201
I20260808 10:08:27.675055 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:27.669359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:40711 (local address 127.2.30.190:38879)
0808 10:08:27.669435 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:27.669438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:27.669684 (+ 246us) server_negotiation.cc:407] Connection header received
0808 10:08:27.670481 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:27.670482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:27.670496 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:27.670572 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:27.671280 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.671847 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.673256 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.673492 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.673559 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:27.674622 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:27.674623 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:27.674624 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:08:27.674629 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:27.674653 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:08:27.674658 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:27.674757 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:27.674890 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 10:08:27.674930 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:08:27.675627 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:27.669242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38879 (local address 127.2.30.130:40711)
0808 10:08:27.669485 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:27.669547 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:27.670334 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:27.670631 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:27.670634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:27.670642 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:27.671189 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:27.671194 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.672024 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.672025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:27.673027 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:27.673030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.673597 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.673598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:27.673806 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:27.674456 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:08:27.674457 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:27.674493 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:08:27.674520 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:27.674758 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:27.674766 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:27.674769 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:08:27.675189 (+ 420us) client_negotiation.cc:240] Negotiation successful
0808 10:08:27.675382 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260808 10:08:27.676137 12097 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38879
I20260808 10:08:27.676237 12097 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:27.676312 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38879
--builtin_ntp_servers=127.2.30.148:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 10:08:27.676524 12097 heartbeater.cc:511] Master 127.2.30.190:38879 requested a full tablet report, sending...
I20260808 10:08:27.676983 11762 ts_manager.cc:194] Registered new tserver with Master: 34430bb0413e4191867cd8bf91732c21 (127.2.30.130:37445)
I20260808 10:08:27.677500 11762 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.30.130:40711
W20260808 10:08:27.771397 12100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:27.771649 12100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:27.771674 12100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:27.773412 12100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:27.773468 12100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:27.773499 12100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:08:27.775264 12100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:27.775563 12100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:27.775928 12100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:27.781845 12109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:27.781894 12106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:27.781894 12107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:27.782371 12100 server_base.cc:1034] running on GCE node
I20260808 10:08:27.782635 12100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:27.782909 12100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:27.784113 12100 hybrid_clock.cc:648] HybridClock initialized: now 1786183707784086 us; error 42 us; skew 500 ppm
I20260808 10:08:27.785887 12100 webserver.cc:460] Webserver started at http://127.2.30.131:44445/ using document root <none> and password file <none>
I20260808 10:08:27.786258 12100 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:27.786363 12100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:27.786552 12100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:27.787978 12100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "3358dd1d15544d4c874923fdaeff6b4b"
format_stamp: "Formatted at 2026-08-08 10:08:27 on dist-test-slave-w206"
server_key: "592b98811fad11c8c5233a1743fc95b1"
server_key_iv: "e8a98f62cac9dd8e007d85588c7da825"
server_key_version: "encryptionkey@0"
I20260808 10:08:27.788432 12100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "3358dd1d15544d4c874923fdaeff6b4b"
format_stamp: "Formatted at 2026-08-08 10:08:27 on dist-test-slave-w206"
server_key: "592b98811fad11c8c5233a1743fc95b1"
server_key_iv: "e8a98f62cac9dd8e007d85588c7da825"
server_key_version: "encryptionkey@0"
I20260808 10:08:27.790175 12100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:08:27.791203 12115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:27.791473 12100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:08:27.791604 12100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "3358dd1d15544d4c874923fdaeff6b4b"
format_stamp: "Formatted at 2026-08-08 10:08:27 on dist-test-slave-w206"
server_key: "592b98811fad11c8c5233a1743fc95b1"
server_key_iv: "e8a98f62cac9dd8e007d85588c7da825"
server_key_version: "encryptionkey@0"
I20260808 10:08:27.791695 12100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:27.800747 12100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:27.801127 12100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:27.801309 12100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:27.801581 12100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:27.801964 12100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:27.802026 12100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:27.802073 12100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:27.802109 12100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:27.808848 12100 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:36393
I20260808 10:08:27.808893 12233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:36393 every 8 connection(s)
I20260808 10:08:27.809294 12100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:08:27.813215 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 12100
I20260808 10:08:27.813362 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:08:27.813580 2170 external_mini_cluster.cc:1428] Setting key 7301b2ab35873be2ef09103d69d6bf9b
I20260808 10:08:27.815868 11809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:27.810218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:48947 (local address 127.2.30.190:38879)
0808 10:08:27.810269 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:27.810272 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:27.810596 (+ 324us) server_negotiation.cc:407] Connection header received
0808 10:08:27.811254 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:27.811256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:27.811273 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:27.811345 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:27.812099 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.812649 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.813929 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.814233 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.814310 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:27.815596 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:27.815598 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:27.815598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:08:27.815602 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:27.815621 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:08:27.815625 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:27.815683 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:27.815765 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 10:08:27.815798 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:08:27.816159 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:27.810200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38879 (local address 127.2.30.131:48947)
0808 10:08:27.810422 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:27.810483 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:27.811117 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:27.811408 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:27.811411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:27.811421 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:27.811989 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:27.811993 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.812775 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.812776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:27.813770 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:27.813774 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:27.814338 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:27.814340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:27.814510 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:27.815407 (+ 897us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:08:27.815408 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:27.815445 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:08:27.815476 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:27.815683 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:27.815686 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:27.815689 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:08:27.815750 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 10:08:27.815854 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260808 10:08:27.816846 12234 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38879
I20260808 10:08:27.816989 12234 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:27.817329 12234 heartbeater.cc:511] Master 127.2.30.190:38879 requested a full tablet report, sending...
I20260808 10:08:27.817860 11762 ts_manager.cc:194] Registered new tserver with Master: 3358dd1d15544d4c874923fdaeff6b4b (127.2.30.131:36393)
I20260808 10:08:27.818300 11762 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.30.131:48947
I20260808 10:08:27.825753 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 10:08:27.828017 11809 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:27.827363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34656 (local address 127.2.30.190:38879)
0808 10:08:27.827446 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:27.827449 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:27.827592 (+ 143us) server_negotiation.cc:407] Connection header received
0808 10:08:27.827625 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:27.827626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:27.827641 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:27.827721 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:27.827897 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34656: BlockingRecv error: recv got EOF from 127.0.0.1:34656 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:08:27.828007 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 11826
I20260808 10:08:27.836004 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 11963
I20260808 10:08:27.841584 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 12100
I20260808 10:08:27.847689 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 11730
2026-08-08T10:08:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (644 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T10:08:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:08:27Z Disabled control of system clock
I20260808 10:08:27.880056 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:33055
--webserver_interface=127.2.30.190
--webserver_port=0
--builtin_ntp_servers=127.2.30.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:33055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 10:08:27.965615 12246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:27.965847 12246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:27.965871 12246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:27.967092 12246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:08:27.967135 12246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:27.967149 12246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:27.967161 12246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:08:27.967173 12246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:08:27.968570 12246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:33055
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:33055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:27.968803 12246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:27.969065 12246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:27.974367 12251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:27.974449 12246 server_base.cc:1034] running on GCE node
W20260808 10:08:27.974383 12252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:27.974370 12254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:27.974861 12246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:27.975209 12246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:27.976393 12246 hybrid_clock.cc:648] HybridClock initialized: now 1786183707976350 us; error 64 us; skew 500 ppm
I20260808 10:08:27.977810 12246 webserver.cc:460] Webserver started at http://127.2.30.190:45753/ using document root <none> and password file <none>
I20260808 10:08:27.978084 12246 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:27.978158 12246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:27.978374 12246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:27.979606 12246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "f073d1a2e493405e97e260b37274163b"
format_stamp: "Formatted at 2026-08-08 10:08:27 on dist-test-slave-w206"
server_key: "e3cfd70bf7914f5a4d0a6e6905692527"
server_key_iv: "271e5c2f810c6220ef7bfa772d7cfb1c"
server_key_version: "encryptionkey@0"
I20260808 10:08:27.979998 12246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "f073d1a2e493405e97e260b37274163b"
format_stamp: "Formatted at 2026-08-08 10:08:27 on dist-test-slave-w206"
server_key: "e3cfd70bf7914f5a4d0a6e6905692527"
server_key_iv: "271e5c2f810c6220ef7bfa772d7cfb1c"
server_key_version: "encryptionkey@0"
I20260808 10:08:27.981525 12246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:08:27.982381 12260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:27.982573 12246 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:27.982687 12246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "f073d1a2e493405e97e260b37274163b"
format_stamp: "Formatted at 2026-08-08 10:08:27 on dist-test-slave-w206"
server_key: "e3cfd70bf7914f5a4d0a6e6905692527"
server_key_iv: "271e5c2f810c6220ef7bfa772d7cfb1c"
server_key_version: "encryptionkey@0"
I20260808 10:08:27.982765 12246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:27.986413 12246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:27.986711 12246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:27.986873 12246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:27.991011 12246 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:33055
I20260808 10:08:27.991077 12312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:33055 every 8 connection(s)
I20260808 10:08:27.991457 12246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:08:27.992244 12313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:27.995266 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 12246
I20260808 10:08:27.995397 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:08:27.995642 2170 external_mini_cluster.cc:1428] Setting key c9e5fd21ddbb6570672044432f430f0d
I20260808 10:08:27.995859 12313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b: Bootstrap starting.
I20260808 10:08:27.996773 12313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:27.997138 12313 log.cc:824] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:27.998104 12313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b: No bootstrap required, opened a new log
I20260808 10:08:27.998677 12316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:27.996347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49206 (local address 127.2.30.190:33055)
0808 10:08:27.996590 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:27.996599 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:27.996627 (+ 28us) server_negotiation.cc:407] Connection header received
0808 10:08:27.997160 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:27.997168 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:27.997216 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:27.997414 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:27.998197 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:27.998201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:27.998202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:08:27.998207 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:27.998263 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:08:27.998279 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:27.998375 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:27.998419 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 10:08:27.998488 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":122,"threads_started":1}
I20260808 10:08:28.000290 12313 raft_consensus.cc:348] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f073d1a2e493405e97e260b37274163b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 33055 } }
I20260808 10:08:28.000483 12313 raft_consensus.cc:374] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:28.000540 12313 raft_consensus.cc:729] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f073d1a2e493405e97e260b37274163b, State: Initialized, Role: FOLLOWER
I20260808 10:08:28.000682 12313 consensus_queue.cc:260] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f073d1a2e493405e97e260b37274163b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 33055 } }
I20260808 10:08:28.001129 12313 raft_consensus.cc:388] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:08:28.001199 12313 raft_consensus.cc:482] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:08:28.001264 12313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:28.002012 12313 raft_consensus.cc:504] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f073d1a2e493405e97e260b37274163b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 33055 } }
I20260808 10:08:28.002179 12313 leader_election.cc:302] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f073d1a2e493405e97e260b37274163b; no voters:
I20260808 10:08:28.002382 12313 leader_election.cc:288] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:08:28.002461 12318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:08:28.002640 12318 raft_consensus.cc:686] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [term 1 LEADER]: Becoming Leader. State: Replica: f073d1a2e493405e97e260b37274163b, State: Running, Role: LEADER
I20260808 10:08:28.002708 12313 sys_catalog.cc:564] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:08:28.002748 12318 consensus_queue.cc:237] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f073d1a2e493405e97e260b37274163b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 33055 } }
I20260808 10:08:28.003211 12319 sys_catalog.cc:455] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f073d1a2e493405e97e260b37274163b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f073d1a2e493405e97e260b37274163b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 33055 } } }
I20260808 10:08:28.003381 12319 sys_catalog.cc:458] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:28.003299 12320 sys_catalog.cc:455] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f073d1a2e493405e97e260b37274163b. Latest consensus state: current_term: 1 leader_uuid: "f073d1a2e493405e97e260b37274163b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f073d1a2e493405e97e260b37274163b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 33055 } } }
I20260808 10:08:28.003561 12320 sys_catalog.cc:458] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:28.004081 12324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:08:28.004705 12324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:08:28.008245 12324 catalog_manager.cc:1347] Generated new cluster ID: 4d5f2fb322c944cb8b056ec750e2ad72
I20260808 10:08:28.008323 12324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:08:28.018797 12324 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:08:28.019086 12324 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:08:28.021488 12324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f073d1a2e493405e97e260b37274163b: Generated new TSK 0
I20260808 10:08:28.021735 12324 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:08:28.026834 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:33055
--builtin_ntp_servers=127.2.30.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 10:08:28.115983 12338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:28.116235 12338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:28.116271 12338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:28.117497 12338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:28.117564 12338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:28.117606 12338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:08:28.119029 12338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:28.119321 12338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:28.119638 12338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:28.124979 12344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:28.124979 12345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:28.125100 12347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:28.125505 12338 server_base.cc:1034] running on GCE node
I20260808 10:08:28.125731 12338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:28.126012 12338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:28.127175 12338 hybrid_clock.cc:648] HybridClock initialized: now 1786183708127160 us; error 33 us; skew 500 ppm
I20260808 10:08:28.128559 12338 webserver.cc:460] Webserver started at http://127.2.30.129:34943/ using document root <none> and password file <none>
I20260808 10:08:28.128842 12338 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:28.128917 12338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:28.129081 12338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:28.130178 12338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "a682baef514f4f28b83fea5f28f9f885"
format_stamp: "Formatted at 2026-08-08 10:08:28 on dist-test-slave-w206"
server_key: "6a23fc17b1837f6c4f0b6036100fb5b4"
server_key_iv: "d58c57a9c550bdd5d7ba9f00fafe3328"
server_key_version: "encryptionkey@0"
I20260808 10:08:28.130543 12338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "a682baef514f4f28b83fea5f28f9f885"
format_stamp: "Formatted at 2026-08-08 10:08:28 on dist-test-slave-w206"
server_key: "6a23fc17b1837f6c4f0b6036100fb5b4"
server_key_iv: "d58c57a9c550bdd5d7ba9f00fafe3328"
server_key_version: "encryptionkey@0"
I20260808 10:08:28.132453 12338 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:08:28.133481 12353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:28.133673 12338 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:28.133764 12338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "a682baef514f4f28b83fea5f28f9f885"
format_stamp: "Formatted at 2026-08-08 10:08:28 on dist-test-slave-w206"
server_key: "6a23fc17b1837f6c4f0b6036100fb5b4"
server_key_iv: "d58c57a9c550bdd5d7ba9f00fafe3328"
server_key_version: "encryptionkey@0"
I20260808 10:08:28.133837 12338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:28.138795 12338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:28.139209 12338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:28.139364 12338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:28.139611 12338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:28.140023 12338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:28.140071 12338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:28.140101 12338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:28.140120 12338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:28.147953 12338 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:33373
I20260808 10:08:28.147991 12471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:33373 every 8 connection(s)
I20260808 10:08:28.148387 12338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:08:28.152045 12316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:28.149379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:54763 (local address 127.2.30.190:33055)
0808 10:08:28.149452 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:28.149453 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:28.149713 (+ 260us) server_negotiation.cc:407] Connection header received
0808 10:08:28.150263 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:28.150265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:28.150271 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:28.150324 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:28.151726 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:28.151728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:28.151729 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:08:28.151734 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:28.151759 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:08:28.151763 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:28.151843 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:28.151945 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 10:08:28.151964 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:08:28.152267 12474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:28.149330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:33055 (local address 127.2.30.129:54763)
0808 10:08:28.149584 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:28.149620 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:28.150138 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:28.150402 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:28.150406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:28.150414 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:28.151552 (+ 1138us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:08:28.151552 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:28.151584 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:08:28.151616 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:28.151841 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:28.151848 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:28.151852 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:08:28.151957 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 10:08:28.152023 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":122,"threads_started":1}
I20260808 10:08:28.152827 12472 heartbeater.cc:347] Connected to a master server at 127.2.30.190:33055
I20260808 10:08:28.152962 12472 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:28.153163 12472 heartbeater.cc:511] Master 127.2.30.190:33055 requested a full tablet report, sending...
I20260808 10:08:28.153352 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 12338
I20260808 10:08:28.153441 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:08:28.153601 2170 external_mini_cluster.cc:1428] Setting key 4009d63d9ba9554665214a1c3a259f9e
I20260808 10:08:28.154279 12276 ts_manager.cc:194] Registered new tserver with Master: a682baef514f4f28b83fea5f28f9f885 (127.2.30.129:33373)
I20260808 10:08:28.154631 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:33055
--builtin_ntp_servers=127.2.30.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 10:08:28.243764 12475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:28.244001 12475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:28.244029 12475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:28.245242 12475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:28.245276 12475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:28.245302 12475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:08:28.246628 12475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:28.246869 12475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:28.247231 12475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:28.252562 12482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:28.252591 12481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:28.252578 12484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:28.252911 12475 server_base.cc:1034] running on GCE node
I20260808 10:08:28.253159 12475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:28.253433 12475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:28.254609 12475 hybrid_clock.cc:648] HybridClock initialized: now 1786183708254596 us; error 30 us; skew 500 ppm
I20260808 10:08:28.256191 12475 webserver.cc:460] Webserver started at http://127.2.30.130:42053/ using document root <none> and password file <none>
I20260808 10:08:28.256533 12475 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:28.256590 12475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:28.256729 12475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:28.257853 12475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "169d2a43ff804c628012033aed3d36b1"
format_stamp: "Formatted at 2026-08-08 10:08:28 on dist-test-slave-w206"
server_key: "052416105b9b7fb3dbb19c48dc08a226"
server_key_iv: "400b59ce1e23014f94b12c5f0ab82e6b"
server_key_version: "encryptionkey@0"
I20260808 10:08:28.258170 12475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "169d2a43ff804c628012033aed3d36b1"
format_stamp: "Formatted at 2026-08-08 10:08:28 on dist-test-slave-w206"
server_key: "052416105b9b7fb3dbb19c48dc08a226"
server_key_iv: "400b59ce1e23014f94b12c5f0ab82e6b"
server_key_version: "encryptionkey@0"
I20260808 10:08:28.259724 12475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:08:28.260627 12490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:28.260900 12475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:28.260980 12475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "169d2a43ff804c628012033aed3d36b1"
format_stamp: "Formatted at 2026-08-08 10:08:28 on dist-test-slave-w206"
server_key: "052416105b9b7fb3dbb19c48dc08a226"
server_key_iv: "400b59ce1e23014f94b12c5f0ab82e6b"
server_key_version: "encryptionkey@0"
I20260808 10:08:28.261089 12475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:28.264813 12475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:28.265105 12475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:28.265250 12475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:28.265468 12475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:28.265800 12475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:28.265832 12475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:28.265885 12475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:28.265913 12475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:28.271744 12475 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:39085
I20260808 10:08:28.271809 12612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:39085 every 8 connection(s)
I20260808 10:08:28.272149 12475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:08:28.275708 12316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:28.272973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:52925 (local address 127.2.30.190:33055)
0808 10:08:28.273107 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:28.273110 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:28.273372 (+ 262us) server_negotiation.cc:407] Connection header received
0808 10:08:28.273953 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:28.273954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:28.273962 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:28.274015 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:28.275372 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:28.275374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:28.275374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:08:28.275380 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:28.275401 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:08:28.275405 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:28.275490 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:28.275604 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 10:08:28.275623 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 10:08:28.275947 12615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:28.272960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:33055 (local address 127.2.30.130:52925)
0808 10:08:28.273219 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:28.273266 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:28.273836 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:28.274086 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:28.274096 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:28.274105 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:28.275180 (+ 1075us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:08:28.275181 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:28.275219 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:08:28.275252 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:28.275486 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:28.275494 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:28.275497 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:08:28.275614 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 10:08:28.275691 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":96,"threads_started":1}
I20260808 10:08:28.276575 12613 heartbeater.cc:347] Connected to a master server at 127.2.30.190:33055
I20260808 10:08:28.276664 12613 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:28.276850 12613 heartbeater.cc:511] Master 127.2.30.190:33055 requested a full tablet report, sending...
I20260808 10:08:28.277350 12276 ts_manager.cc:194] Registered new tserver with Master: 169d2a43ff804c628012033aed3d36b1 (127.2.30.130:39085)
I20260808 10:08:28.280571 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 12475
I20260808 10:08:28.280692 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:08:28.280903 2170 external_mini_cluster.cc:1428] Setting key 2f0e3c3a71b15599f19bb662f622880c
I20260808 10:08:28.282150 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:33055
--builtin_ntp_servers=127.2.30.148:39665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 10:08:28.367967 12616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:28.368234 12616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:28.368276 12616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:28.369521 12616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:28.369591 12616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:28.369633 12616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:08:28.371058 12616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:28.371343 12616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:28.371652 12616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:28.377399 12624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:28.377537 12621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:28.377568 12622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:28.377728 12616 server_base.cc:1034] running on GCE node
I20260808 10:08:28.377951 12616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:28.378172 12616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:28.379357 12616 hybrid_clock.cc:648] HybridClock initialized: now 1786183708379341 us; error 30 us; skew 500 ppm
I20260808 10:08:28.380756 12616 webserver.cc:460] Webserver started at http://127.2.30.131:46005/ using document root <none> and password file <none>
I20260808 10:08:28.381018 12616 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:28.381067 12616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:28.381249 12616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:28.382418 12616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "4e14b9b4e10645e7847eff7f607251cb"
format_stamp: "Formatted at 2026-08-08 10:08:28 on dist-test-slave-w206"
server_key: "a831901b886a1aa260560b5aa50d504e"
server_key_iv: "14de87e92b26e7a0460b6cd6c60664c6"
server_key_version: "encryptionkey@0"
I20260808 10:08:28.382845 12616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "4e14b9b4e10645e7847eff7f607251cb"
format_stamp: "Formatted at 2026-08-08 10:08:28 on dist-test-slave-w206"
server_key: "a831901b886a1aa260560b5aa50d504e"
server_key_iv: "14de87e92b26e7a0460b6cd6c60664c6"
server_key_version: "encryptionkey@0"
I20260808 10:08:28.384353 12616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:08:28.385157 12630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:28.385416 12616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:08:28.385514 12616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "4e14b9b4e10645e7847eff7f607251cb"
format_stamp: "Formatted at 2026-08-08 10:08:28 on dist-test-slave-w206"
server_key: "a831901b886a1aa260560b5aa50d504e"
server_key_iv: "14de87e92b26e7a0460b6cd6c60664c6"
server_key_version: "encryptionkey@0"
I20260808 10:08:28.385591 12616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:28.389212 12616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:28.389550 12616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:28.389694 12616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:28.389919 12616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:28.390244 12616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:28.390306 12616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:28.390347 12616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:28.390375 12616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:28.396593 12616 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:42421
I20260808 10:08:28.396637 12749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:42421 every 8 connection(s)
I20260808 10:08:28.396981 12616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:08:28.398038 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 12616
I20260808 10:08:28.398118 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:08:28.398347 2170 external_mini_cluster.cc:1428] Setting key 821bba31a24030884a7c21708f277a64
I20260808 10:08:28.400832 12316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:28.397785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:49339 (local address 127.2.30.190:33055)
0808 10:08:28.397887 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:28.397889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:28.398218 (+ 329us) server_negotiation.cc:407] Connection header received
0808 10:08:28.399007 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:28.399008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:28.399016 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:28.399090 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:28.400508 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:28.400510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:28.400510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:08:28.400516 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:28.400534 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:08:28.400539 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:28.400606 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:28.400716 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 10:08:28.400733 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 10:08:28.401049 12752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:28.397783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:33055 (local address 127.2.30.131:49339)
0808 10:08:28.398016 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:28.398053 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:28.398838 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:28.399171 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:28.399175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:28.399186 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:28.400328 (+ 1142us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:08:28.400328 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:28.400361 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:08:28.400398 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:28.400607 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:28.400613 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:28.400617 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:08:28.400727 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 10:08:28.400799 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":100,"threads_started":1}
I20260808 10:08:28.401507 12750 heartbeater.cc:347] Connected to a master server at 127.2.30.190:33055
I20260808 10:08:28.401597 12750 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:28.401790 12750 heartbeater.cc:511] Master 127.2.30.190:33055 requested a full tablet report, sending...
I20260808 10:08:28.402158 12276 ts_manager.cc:194] Registered new tserver with Master: 4e14b9b4e10645e7847eff7f607251cb (127.2.30.131:42421)
I20260808 10:08:28.410614 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 10:08:28.412855 12316 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:08:28.412292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49222 (local address 127.2.30.190:33055)
0808 10:08:28.412477 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:28.412479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:28.412521 (+ 42us) server_negotiation.cc:407] Connection header received
0808 10:08:28.412543 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:28.412543 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:28.412548 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:28.412588 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:28.412747 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49222: BlockingRecv error: recv got EOF from 127.0.0.1:49222 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
I20260808 10:08:28.412921 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 12338
I20260808 10:08:28.420145 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 12475
I20260808 10:08:28.425447 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 12616
I20260808 10:08:28.430610 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 12246
2026-08-08T10:08:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (580 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:08:28 dist-test-slave-w206 krb5kdc[12760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:08:28 dist-test-slave-w206 krb5kdc[12760](info): set up 2 sockets
Aug 08 10:08:28 dist-test-slave-w206 krb5kdc[12760](info): commencing operation
krb5kdc: starting...
W20260808 10:08:29.496347 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:08:29 dist-test-slave-w206 krb5kdc[12760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:08:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:08:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:08:29.634621 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:37615
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:46501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:37615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:29.719499 12774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:29.719849 12774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:29.719878 12774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:29.721086 12774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:08:29.721127 12774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:29.721141 12774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:29.721154 12774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:08:29.721166 12774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:08:29.722661 12774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:46501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:37615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:37615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:29.722877 12774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:29.723088 12774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:29.728226 12779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:29.728212 12780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:29.728199 12782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:29.728561 12774 server_base.cc:1034] running on GCE node
I20260808 10:08:29.728766 12774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:29.729096 12774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:29.730374 12774 hybrid_clock.cc:648] HybridClock initialized: now 1786183709730343 us; error 38 us; skew 500 ppm
Aug 08 10:08:29 dist-test-slave-w206 krb5kdc[12760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183709, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:29.733088 12774 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:08:29.733633 12774 webserver.cc:460] Webserver started at http://127.2.30.190:33917/ using document root <none> and password file <none>
I20260808 10:08:29.733882 12774 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:29.733953 12774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:29.734104 12774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:29.735306 12774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "8fbf6f03128f4a918d67e174b2540e00"
format_stamp: "Formatted at 2026-08-08 10:08:29 on dist-test-slave-w206"
server_key: "483d1ca4b59a97d371cd18c8f931ed69"
server_key_iv: "bb4b2bdf7d439ac0365004ada4716596"
server_key_version: "encryptionkey@0"
I20260808 10:08:29.735674 12774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "8fbf6f03128f4a918d67e174b2540e00"
format_stamp: "Formatted at 2026-08-08 10:08:29 on dist-test-slave-w206"
server_key: "483d1ca4b59a97d371cd18c8f931ed69"
server_key_iv: "bb4b2bdf7d439ac0365004ada4716596"
server_key_version: "encryptionkey@0"
I20260808 10:08:29.737116 12774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:08:29.738024 12789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:29.738253 12774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:29.738370 12774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "8fbf6f03128f4a918d67e174b2540e00"
format_stamp: "Formatted at 2026-08-08 10:08:29 on dist-test-slave-w206"
server_key: "483d1ca4b59a97d371cd18c8f931ed69"
server_key_iv: "bb4b2bdf7d439ac0365004ada4716596"
server_key_version: "encryptionkey@0"
I20260808 10:08:29.738451 12774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:29.752957 12774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:29.755626 12774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:29.755820 12774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:29.760568 12774 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:37615
I20260808 10:08:29.760645 12845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:37615 every 8 connection(s)
I20260808 10:08:29.761060 12774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:08:29.762132 12847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:29.764711 12847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00: Bootstrap starting.
I20260808 10:08:29.765358 12847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:29.765681 12847 log.cc:824] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:29.766537 12847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00: No bootstrap required, opened a new log
I20260808 10:08:29.768172 12847 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fbf6f03128f4a918d67e174b2540e00" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37615 } }
I20260808 10:08:29.768325 12847 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:29.768349 12847 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fbf6f03128f4a918d67e174b2540e00, State: Initialized, Role: FOLLOWER
I20260808 10:08:29.768512 12847 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fbf6f03128f4a918d67e174b2540e00" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37615 } }
I20260808 10:08:29.768610 12847 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:08:29.768647 12847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:08:29.768720 12847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:29.769582 12847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fbf6f03128f4a918d67e174b2540e00" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37615 } }
I20260808 10:08:29.769713 12847 leader_election.cc:302] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fbf6f03128f4a918d67e174b2540e00; no voters:
I20260808 10:08:29.769901 12847 leader_election.cc:288] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:08:29.769939 12850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:08:29.770128 12850 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [term 1 LEADER]: Becoming Leader. State: Replica: 8fbf6f03128f4a918d67e174b2540e00, State: Running, Role: LEADER
I20260808 10:08:29.770226 12847 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:08:29.770294 12850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fbf6f03128f4a918d67e174b2540e00" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37615 } }
I20260808 10:08:29.770752 12851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fbf6f03128f4a918d67e174b2540e00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fbf6f03128f4a918d67e174b2540e00" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37615 } } }
I20260808 10:08:29.770865 12851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:29.770913 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 12774
I20260808 10:08:29.771073 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:08:29.771301 2170 external_mini_cluster.cc:1428] Setting key 6217368e9fb0bdf95be732e2d31bc743
I20260808 10:08:29.771284 12852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fbf6f03128f4a918d67e174b2540e00. Latest consensus state: current_term: 1 leader_uuid: "8fbf6f03128f4a918d67e174b2540e00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fbf6f03128f4a918d67e174b2540e00" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37615 } } }
I20260808 10:08:29.771358 12852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00 [sys.catalog]: This master's current role is: LEADER
I20260808 10:08:29.771704 12859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:08:29.772286 12859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:08:29.773957 12859 catalog_manager.cc:1347] Generated new cluster ID: 4b5c1a3430f64eca915517fe296d56dc
I20260808 10:08:29.774016 12859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:08:29 dist-test-slave-w206 krb5kdc[12760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:08:29.783156 12868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:29.771960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36144 (local address 127.2.30.190:37615)
0808 10:08:29.772164 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:29.772173 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:29.772268 (+ 95us) server_negotiation.cc:407] Connection header received
0808 10:08:29.772453 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:29.772459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:29.772505 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:29.772646 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:29.773507 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:29.775782 (+ 2275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:29.776538 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:29.776981 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:29.777102 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:29.779619 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:29.779624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:29.779624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:29.779644 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:29.781450 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:29.781843 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:29.781846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:29.781847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:29.781966 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:29.782183 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:29.782184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:29.782184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:29.782452 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:29.782628 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:29.782699 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 10:08:29.782870 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260808 10:08:29.784556 12859 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:08:29.785089 12859 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:08:29.789458 12859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00: Generated new TSK 0
I20260808 10:08:29.789652 12859 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:08:29.830361 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:37615
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:46501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:29.942790 12873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:29.943051 12873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:29.943081 12873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:29.944233 12873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:29.944280 12873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:29.944308 12873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:08:29.945811 12873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:46501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:37615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:29.946019 12873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:29.946220 12873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:29.951933 12879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:29.951941 12881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:29.951946 12878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:29.951968 12873 server_base.cc:1034] running on GCE node
I20260808 10:08:29.952390 12873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:29.952687 12873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:29.953862 12873 hybrid_clock.cc:648] HybridClock initialized: now 1786183709953849 us; error 31 us; skew 500 ppm
Aug 08 10:08:29 dist-test-slave-w206 krb5kdc[12760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183709, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:29.956758 12873 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:08:29.957255 12873 webserver.cc:460] Webserver started at http://127.2.30.129:43993/ using document root <none> and password file <none>
I20260808 10:08:29.957496 12873 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:29.957566 12873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:29.957721 12873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:29.958815 12873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "e9d3e8e58b8147f188055702e1a16f60"
format_stamp: "Formatted at 2026-08-08 10:08:29 on dist-test-slave-w206"
server_key: "dcdeae5787b806ef9c5795577a5547a5"
server_key_iv: "eb1a04d1464717a3fcee661f19f9b2ea"
server_key_version: "encryptionkey@0"
I20260808 10:08:29.959211 12873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "e9d3e8e58b8147f188055702e1a16f60"
format_stamp: "Formatted at 2026-08-08 10:08:29 on dist-test-slave-w206"
server_key: "dcdeae5787b806ef9c5795577a5547a5"
server_key_iv: "eb1a04d1464717a3fcee661f19f9b2ea"
server_key_version: "encryptionkey@0"
I20260808 10:08:29.960667 12873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:08:29.961421 12888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:29.961603 12873 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:29.961712 12873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "e9d3e8e58b8147f188055702e1a16f60"
format_stamp: "Formatted at 2026-08-08 10:08:29 on dist-test-slave-w206"
server_key: "dcdeae5787b806ef9c5795577a5547a5"
server_key_iv: "eb1a04d1464717a3fcee661f19f9b2ea"
server_key_version: "encryptionkey@0"
I20260808 10:08:29.961787 12873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:29.985587 12873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:29.987952 12873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:29.988138 12873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:29.988441 12873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:29.988821 12873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:29.988881 12873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:29.988919 12873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:29.988965 12873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:29.995105 12873 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:42389
I20260808 10:08:29.995186 13010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:42389 every 8 connection(s)
I20260808 10:08:29.995548 12873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:08:29.997685 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 12873
I20260808 10:08:29.997762 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:08:29.998000 2170 external_mini_cluster.cc:1428] Setting key f6f4847dad922cc5b67dbf7d507f6d8f
Aug 08 10:08:29 dist-test-slave-w206 krb5kdc[12760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183709, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:08:30.005777 12868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:29.996374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:36179 (local address 127.2.30.190:37615)
0808 10:08:29.996429 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:29.996432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:29.996680 (+ 248us) server_negotiation.cc:407] Connection header received
0808 10:08:29.997152 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:29.997153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:29.997168 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:29.997238 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:29.998308 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:29.999022 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.000052 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.000273 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.000352 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.002645 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:30.002647 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:30.002648 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:30.002663 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:30.004260 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:30.004713 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:30.004715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:30.004715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:30.004790 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:30.005058 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:30.005059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:30.005060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:30.005189 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:30.005263 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:30.005564 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 10:08:30.005678 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:08:30.005970 13013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:29.996375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:37615 (local address 127.2.30.129:36179)
0808 10:08:29.996534 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:29.996571 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:29.997060 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:29.997294 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:29.997298 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:29.997952 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:29.998204 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:29.998211 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:29.999114 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:29.999116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:29.999942 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:29.999945 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.000365 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.000366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:30.000496 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.001078 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:30.001088 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:30.002539 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:30.004377 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:30.004388 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:30.004390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:30.004633 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:30.004864 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:30.004865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:30.004865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:30.004989 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:30.005264 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:30.005274 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:30.005422 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 10:08:30.005530 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 10:08:30.005704 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260808 10:08:30.006489 13011 heartbeater.cc:347] Connected to a master server at 127.2.30.190:37615
I20260808 10:08:30.006619 13011 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:30.006865 13011 heartbeater.cc:511] Master 127.2.30.190:37615 requested a full tablet report, sending...
I20260808 10:08:30.007503 12806 ts_manager.cc:194] Registered new tserver with Master: e9d3e8e58b8147f188055702e1a16f60 (127.2.30.129:42389)
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
I20260808 10:08:30.008194 12806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:36179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:08:30.030860 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:37615
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:46501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:30.117926 13016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:30.118145 13016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:30.118170 13016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:30.119329 13016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:30.119364 13016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:30.119390 13016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:08:30.121198 13016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:46501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:37615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:30.121404 13016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:30.121591 13016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:30.126870 13024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:30.126870 13022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:30.126919 13021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:30.127177 13016 server_base.cc:1034] running on GCE node
I20260808 10:08:30.127413 13016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:30.127681 13016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:30.128893 13016 hybrid_clock.cc:648] HybridClock initialized: now 1786183710128872 us; error 29 us; skew 500 ppm
Aug 08 10:08:30 dist-test-slave-w206 krb5kdc[12760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183710, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:30.137401 13016 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:08:30.137928 13016 webserver.cc:460] Webserver started at http://127.2.30.130:37973/ using document root <none> and password file <none>
I20260808 10:08:30.138147 13016 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:30.138190 13016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:30.138298 13016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:30.139703 13016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "d75117c75087404d80387d4409d2ec71"
format_stamp: "Formatted at 2026-08-08 10:08:30 on dist-test-slave-w206"
server_key: "a5cc9b8ac5017d6cccd92d459ac9baab"
server_key_iv: "b04d06347ceea012eed5f204fc3580b5"
server_key_version: "encryptionkey@0"
I20260808 10:08:30.140187 13016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "d75117c75087404d80387d4409d2ec71"
format_stamp: "Formatted at 2026-08-08 10:08:30 on dist-test-slave-w206"
server_key: "a5cc9b8ac5017d6cccd92d459ac9baab"
server_key_iv: "b04d06347ceea012eed5f204fc3580b5"
server_key_version: "encryptionkey@0"
I20260808 10:08:30.142204 13016 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:08:30.143169 13031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:30.143429 13016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:30.143515 13016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "d75117c75087404d80387d4409d2ec71"
format_stamp: "Formatted at 2026-08-08 10:08:30 on dist-test-slave-w206"
server_key: "a5cc9b8ac5017d6cccd92d459ac9baab"
server_key_iv: "b04d06347ceea012eed5f204fc3580b5"
server_key_version: "encryptionkey@0"
I20260808 10:08:30.143570 13016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:30.161202 13016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:30.163615 13016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:30.163739 13016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:30.163986 13016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:30.164355 13016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:30.164407 13016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:30.164431 13016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:30.164443 13016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:30.170718 13016 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:39059
I20260808 10:08:30.170945 13151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:39059 every 8 connection(s)
I20260808 10:08:30.171258 13016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:08:30.177320 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 13016
I20260808 10:08:30.177456 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:08:30.177675 2170 external_mini_cluster.cc:1428] Setting key 8fe6b1a0ef2b5746e6f3076fb0e39081
Aug 08 10:08:30 dist-test-slave-w206 krb5kdc[12760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183710, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:08:30.181658 12868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.172156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:39749 (local address 127.2.30.190:37615)
0808 10:08:30.172203 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:30.172205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:30.172493 (+ 288us) server_negotiation.cc:407] Connection header received
0808 10:08:30.173068 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:30.173070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:30.173085 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:30.173139 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:30.174163 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.174683 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.175779 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.176004 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.176082 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.178552 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:30.178553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:30.178554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:30.178561 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:30.180020 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:30.180560 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:30.180562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:30.180562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:30.180633 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:30.180910 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:30.180911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:30.180911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:30.181068 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:30.181153 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:30.181480 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 10:08:30.181578 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:08:30.181881 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.172206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:37615 (local address 127.2.30.130:39749)
0808 10:08:30.172368 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:30.172407 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:30.172947 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:30.173213 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:30.173217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:30.173840 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:30.174047 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:30.174053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.174788 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.174789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:30.175680 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:30.175682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.176104 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.176105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:30.176233 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.176800 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:30.176808 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:30.178449 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:30.180111 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:30.180119 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:30.180122 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:30.180476 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:30.180707 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:30.180708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:30.180708 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:30.180846 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:30.181174 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:30.181183 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:30.181337 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 10:08:30.181442 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 10:08:30.181597 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260808 10:08:30.182436 13152 heartbeater.cc:347] Connected to a master server at 127.2.30.190:37615
I20260808 10:08:30.182551 13152 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:30.182763 13152 heartbeater.cc:511] Master 127.2.30.190:37615 requested a full tablet report, sending...
I20260808 10:08:30.183166 12806 ts_manager.cc:194] Registered new tserver with Master: d75117c75087404d80387d4409d2ec71 (127.2.30.130:39059)
I20260808 10:08:30.183575 12806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:39749
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:08:30.209934 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:37615
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:46501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:08:30.307314 13157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:08:30.307531 13157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:08:30.307555 13157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:08:30.308754 13157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:08:30.308797 13157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:08:30.308823 13157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:08:30.310318 13157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:46501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:37615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:08:30.310537 13157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:08:30.310734 13157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:08:30.316470 13163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:30.316437 13165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:08:30.316408 13162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:08:30.316421 13157 server_base.cc:1034] running on GCE node
I20260808 10:08:30.316856 13157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:08:30.317180 13157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:08:30.318404 13157 hybrid_clock.cc:648] HybridClock initialized: now 1786183710318389 us; error 34 us; skew 500 ppm
Aug 08 10:08:30 dist-test-slave-w206 krb5kdc[12760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183710, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:08:30.321410 13157 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:08:30.321920 13157 webserver.cc:460] Webserver started at http://127.2.30.131:32811/ using document root <none> and password file <none>
I20260808 10:08:30.322196 13157 fs_manager.cc:359] Metadata directory not provided
I20260808 10:08:30.322247 13157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:08:30.322376 13157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:08:30.323575 13157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "8a796fbb771a4f15a2769f2fcd40dba4"
format_stamp: "Formatted at 2026-08-08 10:08:30 on dist-test-slave-w206"
server_key: "21a943a3533e550618aa1e252b3ff7a6"
server_key_iv: "584578106fbb0cc3aff7d287f8a0c3e8"
server_key_version: "encryptionkey@0"
I20260808 10:08:30.323953 13157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "8a796fbb771a4f15a2769f2fcd40dba4"
format_stamp: "Formatted at 2026-08-08 10:08:30 on dist-test-slave-w206"
server_key: "21a943a3533e550618aa1e252b3ff7a6"
server_key_iv: "584578106fbb0cc3aff7d287f8a0c3e8"
server_key_version: "encryptionkey@0"
I20260808 10:08:30.325552 13157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:08:30.326427 13173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:30.326737 13157 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:08:30.326871 13157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "8a796fbb771a4f15a2769f2fcd40dba4"
format_stamp: "Formatted at 2026-08-08 10:08:30 on dist-test-slave-w206"
server_key: "21a943a3533e550618aa1e252b3ff7a6"
server_key_iv: "584578106fbb0cc3aff7d287f8a0c3e8"
server_key_version: "encryptionkey@0"
I20260808 10:08:30.326959 13157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:08:30.340828 13157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:08:30.343639 13157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:08:30.343825 13157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:08:30.344071 13157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:08:30.344446 13157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:08:30.344506 13157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:30.344551 13157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:08:30.344578 13157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:08:30.351205 13157 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:40765
I20260808 10:08:30.351289 13291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:40765 every 8 connection(s)
I20260808 10:08:30.351662 13157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:08:30.356887 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 13157
I20260808 10:08:30.357100 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:08:30.357352 2170 external_mini_cluster.cc:1428] Setting key 0b83698979147f2c3280340f0115dd8c
Aug 08 10:08:30 dist-test-slave-w206 krb5kdc[12760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183710, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:08:30.363797 12868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.352566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:59577 (local address 127.2.30.190:37615)
0808 10:08:30.352655 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:30.352657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:30.352787 (+ 130us) server_negotiation.cc:407] Connection header received
0808 10:08:30.353883 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:30.353884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:30.353896 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:30.353954 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:30.354668 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.355396 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.356868 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.357084 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.357357 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.360312 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:30.360314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:30.360314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:30.360324 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:30.361982 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:30.362579 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:30.362581 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:30.362581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:30.362653 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:30.362967 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:30.362969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:30.362969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:30.363134 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:30.363225 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:30.363578 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 10:08:30.363677 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 10:08:30.364037 13294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.352466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:37615 (local address 127.2.30.131:59577)
0808 10:08:30.352642 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:30.352684 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:30.353755 (+ 1071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:30.354017 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:30.354020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:30.354322 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:08:30.354539 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:30.354543 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.355529 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.355530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:30.356733 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:30.356738 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.357374 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.357375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:30.357544 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.358303 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:08:30.358315 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:08:30.360124 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:08:30.362128 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:30.362137 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:30.362140 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:30.362473 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:30.362754 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:08:30.362755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:08:30.362755 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:08:30.362883 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:08:30.363237 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:08:30.363245 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:08:30.363407 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 10:08:30.363539 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 10:08:30.363731 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260808 10:08:30.364545 13292 heartbeater.cc:347] Connected to a master server at 127.2.30.190:37615
I20260808 10:08:30.364643 13292 heartbeater.cc:464] Registering TS with master...
I20260808 10:08:30.364809 13292 heartbeater.cc:511] Master 127.2.30.190:37615 requested a full tablet report, sending...
I20260808 10:08:30.365301 12806 ts_manager.cc:194] Registered new tserver with Master: 8a796fbb771a4f15a2769f2fcd40dba4 (127.2.30.131:40765)
I20260808 10:08:30.365805 12806 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:59577
I20260808 10:08:30.369726 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:08:30.378317 12868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.371328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36160 (local address 127.2.30.190:37615)
0808 10:08:30.371384 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:30.371387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:30.371545 (+ 158us) server_negotiation.cc:407] Connection header received
0808 10:08:30.371638 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:30.371639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:30.371650 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:30.371713 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:08:30.372373 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.372919 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.373742 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.373978 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.374071 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.375450 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:08:30.375453 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:08:30.375453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:08:30.375461 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:08:30.376943 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:30.377346 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:30.377348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:30.377348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:30.377415 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:08:30.377625 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:08:30.377626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:08:30.377626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:08:30.377796 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:08:30.377952 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:30.378118 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 10:08:30.378208 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 10:08:30.380249 12806 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:08:30.381902 12806 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:08:30.389493 13305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.384725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:40765 (local address 127.0.0.1:38262)
0808 10:08:30.385253 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:30.385297 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:30.385386 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:30.385706 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:30.385709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:30.385712 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:30.385778 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:30.385780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.387185 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.387187 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:30.388300 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:30.388304 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.389111 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.389112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:30.389211 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.389230 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 10:08:30.389321 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 10:08:30.389386 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":82,"thread_start_us":70,"threads_started":1}
I20260808 10:08:30.389500 13306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.384814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38262 (local address 127.2.30.131:40765)
0808 10:08:30.384991 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:30.384996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:30.385368 (+ 372us) server_negotiation.cc:407] Connection header received
0808 10:08:30.385506 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:30.385509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:30.385526 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:30.385608 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:30.385858 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.387060 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.388385 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.389008 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.389065 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.389105 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:30.389361 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 10:08:30.389408 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":104,"threads_started":1}
I20260808 10:08:30.389956 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.384735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:42389 (local address 127.0.0.1:52866)
0808 10:08:30.385548 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:30.385558 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:30.385610 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:30.385851 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:30.385852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:30.385854 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:30.385909 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:30.385911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.387208 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.387209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:30.388590 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:30.388593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.389580 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.389581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:30.389663 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.389667 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:08:30.389845 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 10:08:30.389888 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"thread_start_us":119,"threads_started":1}
I20260808 10:08:30.390249 13307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.385046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52866 (local address 127.2.30.129:42389)
0808 10:08:30.385392 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:30.385396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:30.385698 (+ 302us) server_negotiation.cc:407] Connection header received
0808 10:08:30.385742 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:30.385744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:30.385757 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:30.385854 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:30.385982 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.387103 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.388702 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.389424 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.389488 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.389542 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:30.390099 (+ 557us) server_negotiation.cc:299] Negotiation successful
0808 10:08:30.390146 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":172,"threads_started":1}
I20260808 10:08:30.390957 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.384735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:39059 (local address 127.0.0.1:55768)
0808 10:08:30.385958 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:30.385968 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:30.386028 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:30.386243 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:30.386244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:30.386246 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:30.386297 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:30.386299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.387458 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.387459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:30.389286 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:30.389289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.390726 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.390727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:30.390783 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.390786 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:08:30.390840 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 10:08:30.390884 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1207,"mutex_wait_us":791,"thread_start_us":115,"threads_started":1}
I20260808 10:08:30.391034 13308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.385037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55768 (local address 127.2.30.130:39059)
0808 10:08:30.385222 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:30.385225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:30.386026 (+ 801us) server_negotiation.cc:407] Connection header received
0808 10:08:30.386114 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:30.386116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:30.386132 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:30.386245 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:30.386372 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.387248 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.389430 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.390564 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.390612 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.390664 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:30.390885 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 10:08:30.390932 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260808 10:08:30.391007 12938 tablet_service.cc:1467] Processing CreateTablet for tablet e1533d50e76e4c1fb13d97494b7b9d40 (DEFAULT_TABLE table=test-table [id=6f6e84293ad84e92995a9bbe3eed6e95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:30.391351 12938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1533d50e76e4c1fb13d97494b7b9d40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:30.391412 13223 tablet_service.cc:1467] Processing CreateTablet for tablet e1533d50e76e4c1fb13d97494b7b9d40 (DEFAULT_TABLE table=test-table [id=6f6e84293ad84e92995a9bbe3eed6e95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:30.391669 13223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1533d50e76e4c1fb13d97494b7b9d40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:30.394258 13309 tablet_bootstrap.cc:492] T e1533d50e76e4c1fb13d97494b7b9d40 P e9d3e8e58b8147f188055702e1a16f60: Bootstrap starting.
I20260808 10:08:30.395002 13310 tablet_bootstrap.cc:492] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4: Bootstrap starting.
I20260808 10:08:30.395056 13309 tablet_bootstrap.cc:654] T e1533d50e76e4c1fb13d97494b7b9d40 P e9d3e8e58b8147f188055702e1a16f60: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:30.395316 13080 tablet_service.cc:1467] Processing CreateTablet for tablet e1533d50e76e4c1fb13d97494b7b9d40 (DEFAULT_TABLE table=test-table [id=6f6e84293ad84e92995a9bbe3eed6e95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:08:30.395651 13309 log.cc:824] T e1533d50e76e4c1fb13d97494b7b9d40 P e9d3e8e58b8147f188055702e1a16f60: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:30.395705 13080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1533d50e76e4c1fb13d97494b7b9d40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:08:30.395743 13310 tablet_bootstrap.cc:654] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:30.396275 13310 log.cc:824] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:30.396575 13309 tablet_bootstrap.cc:492] T e1533d50e76e4c1fb13d97494b7b9d40 P e9d3e8e58b8147f188055702e1a16f60: No bootstrap required, opened a new log
I20260808 10:08:30.396633 13309 ts_tablet_manager.cc:1397] T e1533d50e76e4c1fb13d97494b7b9d40 P e9d3e8e58b8147f188055702e1a16f60: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 10:08:30.397161 13310 tablet_bootstrap.cc:492] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4: No bootstrap required, opened a new log
I20260808 10:08:30.397233 13310 ts_tablet_manager.cc:1397] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:08:30.398598 13313 tablet_bootstrap.cc:492] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71: Bootstrap starting.
I20260808 10:08:30.398731 13309 raft_consensus.cc:348] T e1533d50e76e4c1fb13d97494b7b9d40 P e9d3e8e58b8147f188055702e1a16f60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d75117c75087404d80387d4409d2ec71" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 39059 } } peers { permanent_uuid: "8a796fbb771a4f15a2769f2fcd40dba4" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40765 } } peers { permanent_uuid: "e9d3e8e58b8147f188055702e1a16f60" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 42389 } }
I20260808 10:08:30.398937 13309 raft_consensus.cc:374] T e1533d50e76e4c1fb13d97494b7b9d40 P e9d3e8e58b8147f188055702e1a16f60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:30.399012 13309 raft_consensus.cc:729] T e1533d50e76e4c1fb13d97494b7b9d40 P e9d3e8e58b8147f188055702e1a16f60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9d3e8e58b8147f188055702e1a16f60, State: Initialized, Role: FOLLOWER
I20260808 10:08:30.399138 13309 consensus_queue.cc:260] T e1533d50e76e4c1fb13d97494b7b9d40 P e9d3e8e58b8147f188055702e1a16f60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d75117c75087404d80387d4409d2ec71" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 39059 } } peers { permanent_uuid: "8a796fbb771a4f15a2769f2fcd40dba4" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40765 } } peers { permanent_uuid: "e9d3e8e58b8147f188055702e1a16f60" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 42389 } }
I20260808 10:08:30.399307 13310 raft_consensus.cc:348] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d75117c75087404d80387d4409d2ec71" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 39059 } } peers { permanent_uuid: "8a796fbb771a4f15a2769f2fcd40dba4" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40765 } } peers { permanent_uuid: "e9d3e8e58b8147f188055702e1a16f60" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 42389 } }
I20260808 10:08:30.399448 13309 ts_tablet_manager.cc:1428] T e1533d50e76e4c1fb13d97494b7b9d40 P e9d3e8e58b8147f188055702e1a16f60: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 10:08:30.399466 13310 raft_consensus.cc:374] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:30.399503 13310 raft_consensus.cc:729] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a796fbb771a4f15a2769f2fcd40dba4, State: Initialized, Role: FOLLOWER
I20260808 10:08:30.399571 13011 heartbeater.cc:503] Master 127.2.30.190:37615 was elected leader, sending a full tablet report...
I20260808 10:08:30.399585 13313 tablet_bootstrap.cc:654] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71: Neither blocks nor log segments found. Creating new log.
I20260808 10:08:30.399619 13310 consensus_queue.cc:260] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d75117c75087404d80387d4409d2ec71" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 39059 } } peers { permanent_uuid: "8a796fbb771a4f15a2769f2fcd40dba4" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40765 } } peers { permanent_uuid: "e9d3e8e58b8147f188055702e1a16f60" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 42389 } }
I20260808 10:08:30.399935 13310 ts_tablet_manager.cc:1428] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:08:30.400058 13292 heartbeater.cc:503] Master 127.2.30.190:37615 was elected leader, sending a full tablet report...
I20260808 10:08:30.400169 13313 log.cc:824] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71: Log is configured to *not* fsync() on all Append() calls
I20260808 10:08:30.401269 13313 tablet_bootstrap.cc:492] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71: No bootstrap required, opened a new log
I20260808 10:08:30.401415 13313 ts_tablet_manager.cc:1397] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:08:30.402838 13313 raft_consensus.cc:348] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d75117c75087404d80387d4409d2ec71" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 39059 } } peers { permanent_uuid: "8a796fbb771a4f15a2769f2fcd40dba4" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40765 } } peers { permanent_uuid: "e9d3e8e58b8147f188055702e1a16f60" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 42389 } }
I20260808 10:08:30.402966 13313 raft_consensus.cc:374] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:08:30.403086 13313 raft_consensus.cc:729] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d75117c75087404d80387d4409d2ec71, State: Initialized, Role: FOLLOWER
I20260808 10:08:30.403208 13313 consensus_queue.cc:260] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d75117c75087404d80387d4409d2ec71" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 39059 } } peers { permanent_uuid: "8a796fbb771a4f15a2769f2fcd40dba4" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40765 } } peers { permanent_uuid: "e9d3e8e58b8147f188055702e1a16f60" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 42389 } }
I20260808 10:08:30.403537 13152 heartbeater.cc:503] Master 127.2.30.190:37615 was elected leader, sending a full tablet report...
I20260808 10:08:30.403540 13313 ts_tablet_manager.cc:1428] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260808 10:08:30.421912 13153 tablet.cc:2369] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:08:30.496493 13012 tablet.cc:2369] T e1533d50e76e4c1fb13d97494b7b9d40 P e9d3e8e58b8147f188055702e1a16f60: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:08:30.602347 13293 tablet.cc:2369] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:08:30.698851 13315 raft_consensus.cc:482] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:08:30.699074 13315 raft_consensus.cc:504] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d75117c75087404d80387d4409d2ec71" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 39059 } } peers { permanent_uuid: "8a796fbb771a4f15a2769f2fcd40dba4" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40765 } } peers { permanent_uuid: "e9d3e8e58b8147f188055702e1a16f60" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 42389 } }
I20260808 10:08:30.699512 13315 leader_election.cc:288] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d75117c75087404d80387d4409d2ec71 (127.2.30.130:39059), e9d3e8e58b8147f188055702e1a16f60 (127.2.30.129:42389)
I20260808 10:08:30.703514 13307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.699777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:36815 (local address 127.2.30.129:42389)
0808 10:08:30.699846 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:30.699849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:30.700179 (+ 330us) server_negotiation.cc:407] Connection header received
0808 10:08:30.700214 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:30.700214 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:30.700230 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:30.700284 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:30.700498 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.701162 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.702491 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.703079 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.703142 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.703150 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:30.703403 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 10:08:30.703443 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 10:08:30.703514 13308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.699855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:43133 (local address 127.2.30.130:39059)
0808 10:08:30.699919 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:30.699921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:30.699929 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:08:30.699955 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:30.699956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:30.699964 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:30.700013 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:08:30.700213 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.700995 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.702424 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.703017 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.703111 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.703122 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:30.703396 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 10:08:30.703440 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:08:30.703504 13294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.699710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:39059 (local address 127.2.30.131:43133)
0808 10:08:30.699778 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:30.699791 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:30.699847 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:30.700026 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:30.700027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:30.700029 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:30.700114 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:30.700116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.701089 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.701090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:30.702304 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:30.702308 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.703162 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.703163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:30.703265 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.703269 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:08:30.703335 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:08:30.703392 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":1}
I20260808 10:08:30.703512 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.699725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:42389 (local address 127.2.30.131:36815)
0808 10:08:30.700031 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:30.700041 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:08:30.700106 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:08:30.700343 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:08:30.700343 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:08:30.700345 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:08:30.700413 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:30.700414 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.701237 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.701238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:30.702404 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:08:30.702407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.703162 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.703163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:08:30.703308 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.703312 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:08:30.703376 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 10:08:30.703403 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":76,"thread_start_us":164,"threads_started":1}
I20260808 10:08:30.703883 13103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1533d50e76e4c1fb13d97494b7b9d40" candidate_uuid: "8a796fbb771a4f15a2769f2fcd40dba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d75117c75087404d80387d4409d2ec71" is_pre_election: true
I20260808 10:08:30.704046 13103 raft_consensus.cc:2454] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a796fbb771a4f15a2769f2fcd40dba4 in term 0.
I20260808 10:08:30.704212 12962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1533d50e76e4c1fb13d97494b7b9d40" candidate_uuid: "8a796fbb771a4f15a2769f2fcd40dba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3e8e58b8147f188055702e1a16f60" is_pre_election: true
I20260808 10:08:30.704329 13175 leader_election.cc:302] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a796fbb771a4f15a2769f2fcd40dba4, d75117c75087404d80387d4409d2ec71; no voters:
I20260808 10:08:30.704419 12962 raft_consensus.cc:2454] T e1533d50e76e4c1fb13d97494b7b9d40 P e9d3e8e58b8147f188055702e1a16f60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a796fbb771a4f15a2769f2fcd40dba4 in term 0.
I20260808 10:08:30.704475 13315 raft_consensus.cc:2781] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:08:30.704541 13315 raft_consensus.cc:482] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:08:30.704583 13315 raft_consensus.cc:3037] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:30.705336 13315 raft_consensus.cc:504] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d75117c75087404d80387d4409d2ec71" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 39059 } } peers { permanent_uuid: "8a796fbb771a4f15a2769f2fcd40dba4" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40765 } } peers { permanent_uuid: "e9d3e8e58b8147f188055702e1a16f60" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 42389 } }
I20260808 10:08:30.705502 13315 leader_election.cc:288] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [CANDIDATE]: Term 1 election: Requested vote from peers d75117c75087404d80387d4409d2ec71 (127.2.30.130:39059), e9d3e8e58b8147f188055702e1a16f60 (127.2.30.129:42389)
I20260808 10:08:30.705683 13103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1533d50e76e4c1fb13d97494b7b9d40" candidate_uuid: "8a796fbb771a4f15a2769f2fcd40dba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d75117c75087404d80387d4409d2ec71"
I20260808 10:08:30.705682 12962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1533d50e76e4c1fb13d97494b7b9d40" candidate_uuid: "8a796fbb771a4f15a2769f2fcd40dba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9d3e8e58b8147f188055702e1a16f60"
I20260808 10:08:30.705785 12962 raft_consensus.cc:3037] T e1533d50e76e4c1fb13d97494b7b9d40 P e9d3e8e58b8147f188055702e1a16f60 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:30.705782 13103 raft_consensus.cc:3037] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:08:30.706507 12962 raft_consensus.cc:2454] T e1533d50e76e4c1fb13d97494b7b9d40 P e9d3e8e58b8147f188055702e1a16f60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a796fbb771a4f15a2769f2fcd40dba4 in term 1.
I20260808 10:08:30.706665 13103 raft_consensus.cc:2454] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a796fbb771a4f15a2769f2fcd40dba4 in term 1.
I20260808 10:08:30.706840 13177 leader_election.cc:302] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a796fbb771a4f15a2769f2fcd40dba4, e9d3e8e58b8147f188055702e1a16f60; no voters:
I20260808 10:08:30.706954 13315 raft_consensus.cc:2781] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:08:30.707149 13315 raft_consensus.cc:686] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [term 1 LEADER]: Becoming Leader. State: Replica: 8a796fbb771a4f15a2769f2fcd40dba4, State: Running, Role: LEADER
I20260808 10:08:30.707302 13315 consensus_queue.cc:237] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d75117c75087404d80387d4409d2ec71" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 39059 } } peers { permanent_uuid: "8a796fbb771a4f15a2769f2fcd40dba4" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40765 } } peers { permanent_uuid: "e9d3e8e58b8147f188055702e1a16f60" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 42389 } }
I20260808 10:08:30.708182 12806 catalog_manager.cc:5495] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a796fbb771a4f15a2769f2fcd40dba4 (127.2.30.131). New cstate: current_term: 1 leader_uuid: "8a796fbb771a4f15a2769f2fcd40dba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d75117c75087404d80387d4409d2ec71" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 39059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a796fbb771a4f15a2769f2fcd40dba4" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9d3e8e58b8147f188055702e1a16f60" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 42389 } health_report { overall_health: UNKNOWN } } }
I20260808 10:08:30.747815 13306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.745000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38268 (local address 127.2.30.131:40765)
0808 10:08:30.745074 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 10:08:30.745075 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:08:30.745085 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:08:30.745133 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:08:30.745133 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:08:30.745152 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:08:30.745216 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:08:30.745454 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.745944 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.746785 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:08:30.747105 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:08:30.747171 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:08:30.747331 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:08:30.747489 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:08:30.747558 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:08:30.747647 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 10:08:30.747667 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:08:30.749764 13103 raft_consensus.cc:1261] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71 [term 1 FOLLOWER]: Refusing update from remote peer 8a796fbb771a4f15a2769f2fcd40dba4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:08:30.749755 12962 raft_consensus.cc:1261] T e1533d50e76e4c1fb13d97494b7b9d40 P e9d3e8e58b8147f188055702e1a16f60 [term 1 FOLLOWER]: Refusing update from remote peer 8a796fbb771a4f15a2769f2fcd40dba4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:08:30.750221 13319 consensus_queue.cc:1035] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d75117c75087404d80387d4409d2ec71" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 39059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:08:30.750413 13315 consensus_queue.cc:1035] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9d3e8e58b8147f188055702e1a16f60" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 42389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:08:30.754849 13323 mvcc.cc:205] Tried to move back new op lower bound from 7316208479227932672 to 7316208479057743872. Current Snapshot: MvccSnapshot[applied={T|T < 7316208479227932672}]
I20260808 10:08:30.755131 13324 mvcc.cc:205] Tried to move back new op lower bound from 7316208479227932672 to 7316208479057743872. Current Snapshot: MvccSnapshot[applied={T|T < 7316208479227932672 or (T in {7316208479227932672})}]
I20260808 10:08:30.756386 12806 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:08:30.756505 12806 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:08:30.757426 12806 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8fbf6f03128f4a918d67e174b2540e00: Sending DeleteTablet for 3 replicas of tablet e1533d50e76e4c1fb13d97494b7b9d40
I20260808 10:08:30.757829 13223 tablet_service.cc:1514] Processing DeleteTablet for tablet e1533d50e76e4c1fb13d97494b7b9d40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:30 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:38262
I20260808 10:08:30.757792 12938 tablet_service.cc:1514] Processing DeleteTablet for tablet e1533d50e76e4c1fb13d97494b7b9d40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:30 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:52866
I20260808 10:08:30.758174 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 12873
I20260808 10:08:30.758255 13080 tablet_service.cc:1514] Processing DeleteTablet for tablet e1533d50e76e4c1fb13d97494b7b9d40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:08:30 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:55768
I20260808 10:08:30.760349 13332 tablet_replica.cc:330] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4: stopping tablet replica
I20260808 10:08:30.760458 13332 raft_consensus.cc:2229] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:08:30.760560 13332 raft_consensus.cc:2258] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:30.763571 13332 ts_tablet_manager.cc:1905] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:08:30.765807 13332 ts_tablet_manager.cc:1918] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:08:30.765966 13332 log.cc:1196] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/e1533d50e76e4c1fb13d97494b7b9d40
I20260808 10:08:30.766265 13332 ts_tablet_manager.cc:1939] T e1533d50e76e4c1fb13d97494b7b9d40 P 8a796fbb771a4f15a2769f2fcd40dba4: Deleting consensus metadata
I20260808 10:08:30.766794 12792 catalog_manager.cc:4841] TS 8a796fbb771a4f15a2769f2fcd40dba4 (127.2.30.131:40765): tablet e1533d50e76e4c1fb13d97494b7b9d40 (table test-table [id=6f6e84293ad84e92995a9bbe3eed6e95]) successfully deleted
W20260808 10:08:30.767086 12793 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:42389 (error 108)
I20260808 10:08:30.767118 13333 tablet_replica.cc:330] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71: stopping tablet replica
I20260808 10:08:30.767248 13333 raft_consensus.cc:2229] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:08:30.767387 13333 raft_consensus.cc:2258] T e1533d50e76e4c1fb13d97494b7b9d40 P d75117c75087404d80387d4409d2ec71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:08:30.767699 13305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.767542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:42389 (local address 127.0.0.1:52880)
0808 10:08:30.767613 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:30.767664 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:42389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 10:08:30.767841 12793 catalog_manager.cc:4584] TS e9d3e8e58b8147f188055702e1a16f60 (127.2.30.129:42389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e1533d50e76e4c1fb13d97494b7b9d40: Network error: Client connection negotiation failed: client connection to 127.2.30.129:42389: connect: Connection refused (error 111)
I20260808 10:08:30.768030 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 13016
I20260808 10:08:30.779208 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 13157
I20260808 10:08:30.780532 13305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:08:30.779310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:39059 (local address 127.0.0.1:55772)
0808 10:08:30.780444 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0808 10:08:30.780484 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.130:39059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1097}
I20260808 10:08:30.785377 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 12774
2026-08-08T10:08:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2356 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:08:30 dist-test-slave-w206 krb5kdc[13337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:08:30 dist-test-slave-w206 krb5kdc[13337](info): set up 2 sockets
Aug 08 10:08:30 dist-test-slave-w206 krb5kdc[13337](info): commencing operation
krb5kdc: starting...
W20260808 10:08:31.963627 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:08:32 dist-test-slave-w206 krb5kdc[13337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:08:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:08:32Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.30.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.30.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.30.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/rangeradmin_127.2.30.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.30.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/rangeradmin_127.2.30.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.30.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.30.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.30.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/rangerlookup_127.2.30.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.30.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/rangerlookup_127.2.30.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.30.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.30.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.30.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/HTTP_127.2.30.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.30.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/HTTP_127.2.30.148@KRBTEST.COM.keytab.
I20260808 10:08:32.175213 2170 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskbFyXgX/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/postgres -l logfile start
2026-08-08 10:08:35.349 UTC [13368] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 10:08:35.349 UTC [13368] LOG: listening on IPv4 address "127.2.30.148", port 47047
2026-08-08 10:08:35.355 UTC [13368] LOG: listening on Unix socket "/tmp/.s.PGSQL.47047"
2026-08-08 10:08:35.396 UTC [13372] LOG: database system was shut down at 2026-08-08 10:08:33 UTC
2026-08-08 10:08:35.402 UTC [13368] LOG: database system is ready to accept connections
I20260808 10:08:36.443941 2170 mini_postgres.cc:96] Postgres bound to 47047
2026-08-08 10:08:36.452 UTC [13380] FATAL: database "slave" does not exist
127.2.30.148:47047 - accepting connections
I20260808 10:08:36.453339 2170 mini_ranger.cc:161] Starting Ranger...
I20260808 10:08:36.471897 2170 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 10:08:37.125458 2170 mini_ranger.cc:87] Created ranger Postgres database
I20260808 10:08:37.125564 2170 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-admin
2026-08-08 10:08:37,458 [I] DB FLAVOR :POSTGRES
2026-08-08 10:08:37,458 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:08:37,459 [I] Checking connection..
2026-08-08 10:08:37,459 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 10:08:37,707 [I] Checking connection passed.
2026-08-08 10:08:37,708 [I] --------- Verifying version history table ---------
2026-08-08 10:08:37,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:08:37,980 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:08:37,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:08:38,259 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:08:38,260 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 10:08:38,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 10:08:38.538 UTC [13446] WARNING: there is no transaction in progress
2026-08-08 10:08:38,547 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 10:08:38,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:08:38,814 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 10:08:38,815 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:08:38,815 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 10:08:39,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 10:08:39,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-w206.c.gcp-upstream.internal','N') ;"
2026-08-08 10:08:39,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:08:39,912 [I] Table x_portal_user does not exist in database ranger
2026-08-08 10:08:39,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:08:40,179 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 10:08:40,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:08:40,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:08:40,719 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 10:08:40,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 10:08:41.860 UTC [13563] WARNING: there is no transaction in progress
2026-08-08 10:08:41.879 UTC [13563] WARNING: there is no transaction in progress
2026-08-08 10:08:41.898 UTC [13563] WARNING: there is no transaction in progress
2026-08-08 10:08:41.917 UTC [13563] WARNING: there is no transaction in progress
2026-08-08 10:08:41.938 UTC [13563] WARNING: there is no transaction in progress
2026-08-08 10:08:41.962 UTC [13563] WARNING: there is no transaction in progress
2026-08-08 10:08:42.031 UTC [13563] WARNING: there is no transaction in progress
2026-08-08 10:08:42.040 UTC [13563] WARNING: there is no transaction in progress
2026-08-08 10:08:42.058 UTC [13563] WARNING: there is no transaction in progress
2026-08-08 10:08:42.071 UTC [13563] WARNING: there is no transaction in progress
2026-08-08 10:08:42.084 UTC [13563] WARNING: there is no transaction in progress
2026-08-08 10:08:42.101 UTC [13563] WARNING: there is no transaction in progress
2026-08-08 10:08:42.112 UTC [13563] WARNING: there is no transaction in progress
2026-08-08 10:08:42.121 UTC [13563] WARNING: there is no transaction in progress
2026-08-08 10:08:42.131 UTC [13563] WARNING: there is no transaction in progress
2026-08-08 10:08:42.140 UTC [13563] WARNING: there is no transaction in progress
2026-08-08 10:08:42,983 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 10:08:42,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 10:08:43,247 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 10:08:43,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:08:43,498 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 10:08:43,499 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:08:43,789 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 10:08:43,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:08:44,082 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:08:44,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-w206.c.gcp-upstream.internal';"
2026-08-08 10:08:44,625 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 10:08:44,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 10:08:44,890 [I] DB_PATCHES have already been applied
I20260808 10:08:44.896879 2170 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-admin:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskbFyXgX/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 10:08:44.896961 2170 mini_ranger.cc:193] Using host: 127.2.30.148
I20260808 10:08:44.901710 2170 mini_ranger.cc:227] Ranger admin URL: http://127.2.30.148:34895
Aug 08, 2026 10:08:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:08:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:08:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 10:08:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 10:08:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 10:08:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 10:08:45 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 10:08:46.077072 2170 mini_ranger.cc:160] Time spent starting Ranger: real 9.624s user 0.000s sys 0.013s
Aug 08, 2026 10:09:08 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 10:09:08 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 08, 2026 10:09:08 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 10:09:08 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 10:09:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 10:09:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 10:09:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 10:09:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 10:09:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 10:09:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 10:09:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 10:09:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 10:09:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 10:09:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 10:09:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 10:09:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 10:09:08 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 10:09:09 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 10:09:09.859619 2170 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.30.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.30.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.30.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/rangerkms_127.2.30.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.30.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/rangerkms_127.2.30.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.30.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.30.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.30.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/HTTP_127.2.30.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.30.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/HTTP_127.2.30.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 10:09:09 dist-test-slave-w206 krb5kdc[13337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183749, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 10:09:09.971508 2170 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 10:09:09.987771 2170 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 10:09:11.187906 2170 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 10:09:11.188012 2170 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-kms
I20260808 10:09:11.188040 2170 mini_ranger_kms.cc:227] Using postgres at 127.2.30.148:47047
2026-08-08 10:09:11,945 [I] DB FLAVOR :POSTGRES
2026-08-08 10:09:11,946 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:09:11,946 [I] Checking connection
2026-08-08 10:09:11,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 10:09:12,286 [I] connection success
2026-08-08 10:09:12,286 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 10:09:12,286 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 10:09:12,287 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 10:09:12,559 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 10:09:12,559 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:09:12,560 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 10:09:12,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:47047/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 10:09:12,848 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 10:09:13.428942 2170 mini_ranger_kms.cc:310] Created kms service
I20260808 10:09:13.617682 2170 mini_ranger_kms.cc:326] Created kudu user
I20260808 10:09:13.662936 2170 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 10:09:13 dist-test-slave-w206 krb5kdc[13337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183749, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
I20260808 10:09:14.137252 2170 mini_ranger_kms.cc:384] Added ranger policy
I20260808 10:09:14.137387 2170 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-kms:/tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 10:09:14.137410 2170 mini_ranger_kms.cc:240] Using host: 127.2.30.148
I20260808 10:09:14.141814 2170 mini_ranger_kms.cc:276] Ranger KMS PID: 13752
I20260808 10:09:14.141952 2170 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.30.148:58121
Aug 08, 2026 10:09:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:09:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:09:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 10:09:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 10:09:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 10:09:16 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 10:09:16.588557 2170 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.617s user 0.000s sys 0.727s
I20260808 10:09:16.588676 2170 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 10:09:16.588761 2170 mini_ranger_kms.cc:401] 127.2.30.148:58121/kms/v1/keys
Aug 08 10:09:17 dist-test-slave-w206 krb5kdc[13337](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786183757, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.30.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 10:09:17 dist-test-slave-w206 krb5kdc[13337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183757, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.30.148@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
Aug 08 10:09:18 dist-test-slave-w206 krb5kdc[13337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183757, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.30.148@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
Aug 08 10:09:21 dist-test-slave-w206 krb5kdc[13337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:09:21.102663 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:43609
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:45921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:43609
--ranger_config_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.30.148:58121/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:09:21.229638 13805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:09:21.229943 13805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:09:21.229987 13805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:09:21.231597 13805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:09:21.231693 13805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:09:21.231719 13805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:09:21.231740 13805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:09:21.231789 13805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:09:21.234194 13805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:45921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.30.148:58121/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:43609
--ranger_config_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:43609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:09:21.234561 13805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:09:21.234848 13805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:09:21.242878 13805 server_base.cc:1034] running on GCE node
W20260808 10:09:21.242825 13811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:09:21.242820 13813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:09:21.243001 13810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:09:21.243523 13805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:09:21.243877 13805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:09:21.245059 13805 hybrid_clock.cc:648] HybridClock initialized: now 1786183761245052 us; error 54 us; skew 500 ppm
Aug 08 10:09:21 dist-test-slave-w206 krb5kdc[13337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183761, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:09:21.248668 13805 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:09:21.249185 13805 webserver.cc:460] Webserver started at http://127.2.30.190:33591/ using document root <none> and password file <none>
I20260808 10:09:21.249401 13805 fs_manager.cc:359] Metadata directory not provided
I20260808 10:09:21.249440 13805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:09:21.249542 13805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:09:21 dist-test-slave-w206 krb5kdc[13337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183761, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
I20260808 10:09:21.313575 13805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "7f00d5a240e14359a85cf9a5b06ce63f"
format_stamp: "Formatted at 2026-08-08 10:09:21 on dist-test-slave-w206"
server_key: "abf38a323a2bd836dea6b048d55cd160"
server_key_iv: "bcb50af479695982d68a41b7c3305e78"
server_key_version: "kuduclusterkey@0"
I20260808 10:09:21.314116 13805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "7f00d5a240e14359a85cf9a5b06ce63f"
format_stamp: "Formatted at 2026-08-08 10:09:21 on dist-test-slave-w206"
server_key: "abf38a323a2bd836dea6b048d55cd160"
server_key_iv: "bcb50af479695982d68a41b7c3305e78"
server_key_version: "kuduclusterkey@0"
I20260808 10:09:21.316097 13805 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 10:09:21.333302 13821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:09:21.333572 13805 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 10:09:21.333699 13805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "7f00d5a240e14359a85cf9a5b06ce63f"
format_stamp: "Formatted at 2026-08-08 10:09:21 on dist-test-slave-w206"
server_key: "abf38a323a2bd836dea6b048d55cd160"
server_key_iv: "bcb50af479695982d68a41b7c3305e78"
server_key_version: "kuduclusterkey@0"
I20260808 10:09:21.333813 13805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:09:21.362689 13805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:09:21.365438 13805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:09:21.365646 13805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:09:21.377731 13805 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:43609
I20260808 10:09:21.378065 13875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:43609 every 8 connection(s)
I20260808 10:09:21.378288 13805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:09:21.379601 13876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:09:21.382061 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 13805
I20260808 10:09:21.382175 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:09:21.382373 2170 external_mini_cluster.cc:1428] Setting key 81d9a0181001f21cf48c9a62ff76fb4a
I20260808 10:09:21.384563 13876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f: Bootstrap starting.
I20260808 10:09:21.385993 13876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f: Neither blocks nor log segments found. Creating new log.
I20260808 10:09:21.387163 13876 log.cc:824] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:09:21.388244 13876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f: No bootstrap required, opened a new log
Aug 08 10:09:21 dist-test-slave-w206 krb5kdc[13337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:09:21.392995 13876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f00d5a240e14359a85cf9a5b06ce63f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43609 } }
I20260808 10:09:21.393201 13876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:09:21.393245 13876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f00d5a240e14359a85cf9a5b06ce63f, State: Initialized, Role: FOLLOWER
I20260808 10:09:21.393471 13876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f00d5a240e14359a85cf9a5b06ce63f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43609 } }
I20260808 10:09:21.393585 13876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:09:21.393683 13876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:09:21.393754 13876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:09:21.394976 13876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f00d5a240e14359a85cf9a5b06ce63f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43609 } }
I20260808 10:09:21.395156 13876 leader_election.cc:302] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f00d5a240e14359a85cf9a5b06ce63f; no voters:
I20260808 10:09:21.395445 13876 leader_election.cc:288] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:09:21.395896 13882 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:09:21.395911 13876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:09:21.396215 13882 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [term 1 LEADER]: Becoming Leader. State: Replica: 7f00d5a240e14359a85cf9a5b06ce63f, State: Running, Role: LEADER
I20260808 10:09:21.396325 13882 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f00d5a240e14359a85cf9a5b06ce63f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43609 } }
I20260808 10:09:21.398635 13881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f00d5a240e14359a85cf9a5b06ce63f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f00d5a240e14359a85cf9a5b06ce63f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43609 } } }
I20260808 10:09:21.398926 13881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [sys.catalog]: This master's current role is: LEADER
I20260808 10:09:21.399063 13879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:21.383079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52552 (local address 127.2.30.190:43609)
0808 10:09:21.383504 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 10:09:21.383516 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:09:21.383600 (+ 84us) server_negotiation.cc:407] Connection header received
0808 10:09:21.384615 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:09:21.384625 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:09:21.384693 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:09:21.384860 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:09:21.385679 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:21.387829 (+ 2150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:21.389052 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:21.390161 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:21.390324 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:21.393842 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:09:21.393846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:09:21.393847 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:09:21.393862 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:09:21.396842 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:09:21.397535 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:09:21.397538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:09:21.397538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:09:21.397656 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:09:21.397919 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:09:21.397920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:09:21.397921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:09:21.398260 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:09:21.398484 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 10:09:21.398615 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 10:09:21.398764 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":175,"threads_started":1}
I20260808 10:09:21.401799 13876 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 10:09:21.402427 13897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:09:21.407094 13883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f00d5a240e14359a85cf9a5b06ce63f. Latest consensus state: current_term: 1 leader_uuid: "7f00d5a240e14359a85cf9a5b06ce63f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f00d5a240e14359a85cf9a5b06ce63f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43609 } } }
I20260808 10:09:21.407646 13883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f [sys.catalog]: This master's current role is: LEADER
I20260808 10:09:21.408866 13897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:09:21.415751 13897 catalog_manager.cc:1347] Generated new cluster ID: 1308ee0009324ebda153889d00ffae57
I20260808 10:09:21.415867 13897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:09:21.448776 13897 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:09:21.449638 13897 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:09:21.454450 13897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f: Generated new TSK 0
I20260808 10:09:21.454684 13897 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:09:22.391443 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.30.148:58121/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43609
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:45921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:09:22.533422 13924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:09:22.533654 13924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:09:22.533681 13924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:09:22.535012 13924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:09:22.535059 13924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:09:22.535089 13924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:09:22.536896 13924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:45921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.30.148:58121/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:09:22.537191 13924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:09:22.537488 13924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:09:22.543823 13933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:09:22.543874 13930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:09:22.544394 13931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:09:22.545423 13924 server_base.cc:1034] running on GCE node
I20260808 10:09:22.545673 13924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:09:22.545960 13924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:09:22.547143 13924 hybrid_clock.cc:648] HybridClock initialized: now 1786183762547125 us; error 32 us; skew 500 ppm
Aug 08 10:09:22 dist-test-slave-w206 krb5kdc[13337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183762, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:09:22.550665 13924 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:09:22.551373 13924 webserver.cc:460] Webserver started at http://127.2.30.129:38129/ using document root <none> and password file <none>
I20260808 10:09:22.551702 13924 fs_manager.cc:359] Metadata directory not provided
I20260808 10:09:22.551782 13924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:09:22.551940 13924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:09:22 dist-test-slave-w206 krb5kdc[13337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183762, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
I20260808 10:09:22.570322 13924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "9394be1c3c48471d86d9416ffc6c9b6d"
format_stamp: "Formatted at 2026-08-08 10:09:22 on dist-test-slave-w206"
server_key: "87a9de441f90136e0fdfc4adc450fa7a"
server_key_iv: "9a975ca116615bc4587b35fdfaf6e637"
server_key_version: "kuduclusterkey@0"
I20260808 10:09:22.570880 13924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "9394be1c3c48471d86d9416ffc6c9b6d"
format_stamp: "Formatted at 2026-08-08 10:09:22 on dist-test-slave-w206"
server_key: "87a9de441f90136e0fdfc4adc450fa7a"
server_key_iv: "9a975ca116615bc4587b35fdfaf6e637"
server_key_version: "kuduclusterkey@0"
I20260808 10:09:22.573033 13924 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260808 10:09:22.588967 13940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:09:22.589218 13924 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:09:22.589330 13924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "9394be1c3c48471d86d9416ffc6c9b6d"
format_stamp: "Formatted at 2026-08-08 10:09:22 on dist-test-slave-w206"
server_key: "87a9de441f90136e0fdfc4adc450fa7a"
server_key_iv: "9a975ca116615bc4587b35fdfaf6e637"
server_key_version: "kuduclusterkey@0"
I20260808 10:09:22.589435 13924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:09:22.613785 13924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:09:22.616842 13924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:09:22.617044 13924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:09:22.617522 13924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:09:22.618062 13924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:09:22.618144 13924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:09:22.618199 13924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:09:22.618233 13924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:09:22.629972 13924 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:37303
I20260808 10:09:22.630640 13924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:09:22.631189 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 13924
I20260808 10:09:22.631295 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:09:22.631518 2170 external_mini_cluster.cc:1428] Setting key ad83f46e35ba394425f5ee87ee7ad050
I20260808 10:09:22.642992 14060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:37303 every 8 connection(s)
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Aug 08 10:09:22 dist-test-slave-w206 krb5kdc[13337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183762, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:09:22.669059 14065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:22.646033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:43609 (local address 127.2.30.129:42177)
0808 10:09:22.647574 (+ 1541us) negotiation.cc:107] Waiting for socket to connect
0808 10:09:22.647632 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:09:22.648428 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:09:22.648897 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:09:22.648902 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:09:22.653075 (+ 4173us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:09:22.653581 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:09:22.653593 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:09:22.655270 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:09:22.655271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:09:22.660317 (+ 5046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:09:22.660326 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:09:22.661077 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:09:22.661078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:09:22.661266 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:22.662017 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:09:22.662030 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:09:22.664305 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:09:22.666861 (+ 2556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:09:22.666872 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:09:22.666876 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:09:22.667290 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:09:22.667629 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:09:22.667630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:09:22.667631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:09:22.667812 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:09:22.668192 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:09:22.668201 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:09:22.668412 (+ 211us) client_negotiation.cc:769] Sending connection context
0808 10:09:22.668523 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 10:09:22.668707 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1474,"thread_start_us":132,"threads_started":1}
I20260808 10:09:22.669780 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:22.646423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:42177 (local address 127.2.30.190:43609)
0808 10:09:22.648637 (+ 2214us) server_negotiation.cc:206] Beginning negotiation
0808 10:09:22.648639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:09:22.648648 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:09:22.648674 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:09:22.648675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:09:22.648690 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:09:22.648749 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:09:22.654028 (+ 5279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:22.655019 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:22.660531 (+ 5512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:22.660886 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:22.660972 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:22.664473 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:09:22.664475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:09:22.664476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:09:22.664485 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:09:22.666740 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:09:22.667429 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:09:22.667431 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:09:22.667432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:09:22.667516 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:09:22.667918 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:09:22.667920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:09:22.667920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:09:22.668083 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:09:22.668153 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 10:09:22.669523 (+ 1370us) server_negotiation.cc:299] Negotiation successful
0808 10:09:22.669639 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2186,"thread_start_us":71,"threads_started":1}
I20260808 10:09:22.670787 14061 heartbeater.cc:347] Connected to a master server at 127.2.30.190:43609
I20260808 10:09:22.670915 14061 heartbeater.cc:464] Registering TS with master...
I20260808 10:09:22.671168 14061 heartbeater.cc:511] Master 127.2.30.190:43609 requested a full tablet report, sending...
I20260808 10:09:22.671895 13838 ts_manager.cc:194] Registered new tserver with Master: 9394be1c3c48471d86d9416ffc6c9b6d (127.2.30.129:37303)
I20260808 10:09:22.672914 13838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:42177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:09:22.687536 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.30.148:58121/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43609
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:45921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:09:22.847978 14068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:09:22.848412 14068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:09:22.848510 14068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:09:22.850800 14068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:09:22.850962 14068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:09:22.851123 14068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:09:22.854127 14068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:45921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.30.148:58121/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:09:22.854758 14068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:09:22.855225 14068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:09:22.865178 14075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:09:22.865581 14068 server_base.cc:1034] running on GCE node
W20260808 10:09:22.866358 14079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:09:22.867117 14077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:09:22.867542 14068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:09:22.867898 14068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:09:22.875294 14068 hybrid_clock.cc:648] HybridClock initialized: now 1786183762875262 us; error 47 us; skew 500 ppm
Aug 08 10:09:22 dist-test-slave-w206 krb5kdc[13337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183762, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:09:22.880982 14068 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:09:22.881783 14068 webserver.cc:460] Webserver started at http://127.2.30.130:39273/ using document root <none> and password file <none>
I20260808 10:09:22.882130 14068 fs_manager.cc:359] Metadata directory not provided
I20260808 10:09:22.882270 14068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:09:22.883195 14068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:09:22 dist-test-slave-w206 krb5kdc[13337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183762, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
Aug 08 10:09:22 dist-test-slave-w206 krb5kdc[13337](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786183762, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:09:22.904209 14068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "ebe6e071121c47b9b2aaa4a98751a394"
format_stamp: "Formatted at 2026-08-08 10:09:22 on dist-test-slave-w206"
server_key: "efb5b1f5fe615a460226a9215c339ece"
server_key_iv: "7e5a0ddcb6f952e5f0973307ed83d157"
server_key_version: "kuduclusterkey@0"
I20260808 10:09:22.904942 14068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "ebe6e071121c47b9b2aaa4a98751a394"
format_stamp: "Formatted at 2026-08-08 10:09:22 on dist-test-slave-w206"
server_key: "efb5b1f5fe615a460226a9215c339ece"
server_key_iv: "7e5a0ddcb6f952e5f0973307ed83d157"
server_key_version: "kuduclusterkey@0"
I20260808 10:09:22.907500 14068 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260808 10:09:22.934113 14086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:09:22.934435 14068 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:09:22.934580 14068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "ebe6e071121c47b9b2aaa4a98751a394"
format_stamp: "Formatted at 2026-08-08 10:09:22 on dist-test-slave-w206"
server_key: "efb5b1f5fe615a460226a9215c339ece"
server_key_iv: "7e5a0ddcb6f952e5f0973307ed83d157"
server_key_version: "kuduclusterkey@0"
I20260808 10:09:22.934710 14068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:09:22.958091 14068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:09:22.961524 14068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:09:22.961849 14068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:09:22.962216 14068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:09:22.962720 14068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:09:22.962832 14068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:09:22.962913 14068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:09:22.963017 14068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:09:22.976778 14068 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:35653
I20260808 10:09:22.977013 14204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:35653 every 8 connection(s)
I20260808 10:09:22.977435 14068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:09:22.977896 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 14068
I20260808 10:09:22.978013 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:09:22.978224 2170 external_mini_cluster.cc:1428] Setting key c59f9bdfd44b706c280c830b7619b4e4
Aug 08 10:09:22 dist-test-slave-w206 krb5kdc[13337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183762, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:09:22.997254 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:22.983352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:55173 (local address 127.2.30.190:43609)
0808 10:09:22.983406 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 10:09:22.983408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:09:22.983742 (+ 334us) server_negotiation.cc:407] Connection header received
0808 10:09:22.984786 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:09:22.984788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:09:22.984805 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:09:22.984848 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:09:22.986064 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:22.986813 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:22.988617 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:22.989048 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:22.989138 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:22.993007 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:09:22.993008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:09:22.993009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:09:22.993019 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:09:22.994958 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:09:22.995636 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:09:22.995638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:09:22.995638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:09:22.995710 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:09:22.996122 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:09:22.996123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:09:22.996123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:09:22.996316 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:09:22.996406 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 10:09:22.997013 (+ 607us) server_negotiation.cc:299] Negotiation successful
0808 10:09:22.997139 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:09:22.998246 14208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:22.982960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:43609 (local address 127.2.30.130:55173)
0808 10:09:22.983534 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0808 10:09:22.983603 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 10:09:22.984645 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:09:22.985061 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:09:22.985066 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:09:22.985476 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:09:22.985934 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:09:22.985942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:09:22.987039 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:09:22.987040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:09:22.988481 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:09:22.988485 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:09:22.989317 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:09:22.989318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:09:22.989592 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:22.990476 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:09:22.990488 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:09:22.992848 (+ 2360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:09:22.995147 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:09:22.995158 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:09:22.995172 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:09:22.995533 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:09:22.995782 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:09:22.995783 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:09:22.995783 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:09:22.996033 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:09:22.996475 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:09:22.996483 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:09:22.996779 (+ 296us) client_negotiation.cc:769] Sending connection context
0808 10:09:22.997665 (+ 886us) client_negotiation.cc:240] Negotiation successful
0808 10:09:22.997924 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":161,"threads_started":1}
I20260808 10:09:22.999074 14205 heartbeater.cc:347] Connected to a master server at 127.2.30.190:43609
I20260808 10:09:22.999195 14205 heartbeater.cc:464] Registering TS with master...
I20260808 10:09:22.999425 14205 heartbeater.cc:511] Master 127.2.30.190:43609 requested a full tablet report, sending...
I20260808 10:09:22.999929 13838 ts_manager.cc:194] Registered new tserver with Master: ebe6e071121c47b9b2aaa4a98751a394 (127.2.30.130:35653)
I20260808 10:09:23.000434 13838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:55173
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:09:23.040272 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.30.148:58121/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43609
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:45921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:09:23.202970 14212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:09:23.203475 14212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:09:23.203590 14212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:09:23.205974 14212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:09:23.206115 14212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:09:23.206212 14212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:09:23.209455 14212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:45921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.30.148:58121/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43609
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:09:23.210124 14212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:09:23.210553 14212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:09:23.219067 14218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:09:23.219391 14217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:09:23.220111 14220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:09:23.228830 14212 server_base.cc:1034] running on GCE node
I20260808 10:09:23.229272 14212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:09:23.229671 14212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:09:23.230845 14212 hybrid_clock.cc:648] HybridClock initialized: now 1786183763230821 us; error 46 us; skew 500 ppm
Aug 08 10:09:23 dist-test-slave-w206 krb5kdc[13337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183763, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:09:23.235232 14212 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:09:23.236047 14212 webserver.cc:460] Webserver started at http://127.2.30.131:42299/ using document root <none> and password file <none>
I20260808 10:09:23.236407 14212 fs_manager.cc:359] Metadata directory not provided
I20260808 10:09:23.236586 14212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:09:23.236831 14212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:09:23 dist-test-slave-w206 krb5kdc[13337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183763, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
I20260808 10:09:23.269481 14212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "558b5ef6c9cf49bb939a26c3f5ec482b"
format_stamp: "Formatted at 2026-08-08 10:09:23 on dist-test-slave-w206"
server_key: "6027d01864e8c01ad3b3b619df10bcf9"
server_key_iv: "3cf0f8062ef4cdac776393edbf7d4f37"
server_key_version: "kuduclusterkey@0"
I20260808 10:09:23.270234 14212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "558b5ef6c9cf49bb939a26c3f5ec482b"
format_stamp: "Formatted at 2026-08-08 10:09:23 on dist-test-slave-w206"
server_key: "6027d01864e8c01ad3b3b619df10bcf9"
server_key_iv: "3cf0f8062ef4cdac776393edbf7d4f37"
server_key_version: "kuduclusterkey@0"
I20260808 10:09:23.276029 14212 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.002s
I20260808 10:09:23.291296 14227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:09:23.291607 14212 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:09:23.291786 14212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "558b5ef6c9cf49bb939a26c3f5ec482b"
format_stamp: "Formatted at 2026-08-08 10:09:23 on dist-test-slave-w206"
server_key: "6027d01864e8c01ad3b3b619df10bcf9"
server_key_iv: "3cf0f8062ef4cdac776393edbf7d4f37"
server_key_version: "kuduclusterkey@0"
I20260808 10:09:23.292007 14212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:09:23.314677 14212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:09:23.317955 14212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:09:23.318157 14212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:09:23.318554 14212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:09:23.320833 14212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:09:23.320937 14212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:09:23.320994 14212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:09:23.321048 14212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:09:23.336167 14212 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:35329
I20260808 10:09:23.337023 14212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:09:23.339795 14346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:35329 every 8 connection(s)
I20260808 10:09:23.346735 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 14212
I20260808 10:09:23.346858 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:09:23.347271 2170 external_mini_cluster.cc:1428] Setting key 4a0dfa324ec2ea30f9999c33f53a96d3
Aug 08 10:09:23 dist-test-slave-w206 krb5kdc[13337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183763, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:09:23.355253 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:23.338749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:34881 (local address 127.2.30.190:43609)
0808 10:09:23.338805 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 10:09:23.338808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:09:23.338827 (+ 19us) server_negotiation.cc:407] Connection header received
0808 10:09:23.340416 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:09:23.340417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:09:23.340434 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:09:23.340513 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:09:23.342028 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.342886 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.345339 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.345658 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.345760 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:23.350249 (+ 4489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:09:23.350251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:09:23.350251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:09:23.350272 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:09:23.352565 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:09:23.353574 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:09:23.353576 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:09:23.353577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:09:23.353655 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:09:23.354163 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:09:23.354164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:09:23.354165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:09:23.354331 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:09:23.354408 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:09:23.355028 (+ 620us) server_negotiation.cc:299] Negotiation successful
0808 10:09:23.355148 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 10:09:23.355850 14349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:23.338238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:43609 (local address 127.2.30.131:34881)
0808 10:09:23.338630 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 10:09:23.338693 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 10:09:23.340155 (+ 1462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:09:23.340655 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:09:23.340661 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:09:23.341315 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:09:23.341861 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:09:23.341870 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.343159 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.343160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:09:23.345153 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:09:23.345158 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.345804 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.345805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:09:23.346120 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:23.347365 (+ 1245us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:09:23.347391 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:09:23.350106 (+ 2715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:09:23.352705 (+ 2599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:09:23.352732 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:09:23.352736 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:09:23.353416 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:09:23.353746 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:09:23.353747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:09:23.353747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:09:23.354055 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:09:23.354436 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:09:23.354445 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:09:23.354756 (+ 311us) client_negotiation.cc:769] Sending connection context
0808 10:09:23.354963 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 10:09:23.355355 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":241,"threads_started":1}
I20260808 10:09:23.356796 14347 heartbeater.cc:347] Connected to a master server at 127.2.30.190:43609
I20260808 10:09:23.357046 14347 heartbeater.cc:464] Registering TS with master...
I20260808 10:09:23.357410 14347 heartbeater.cc:511] Master 127.2.30.190:43609 requested a full tablet report, sending...
I20260808 10:09:23.358057 13838 ts_manager.cc:194] Registered new tserver with Master: 558b5ef6c9cf49bb939a26c3f5ec482b (127.2.30.131:35329)
I20260808 10:09:23.358596 13838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:34881
I20260808 10:09:23.359449 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:09:23.379087 14066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:23.363508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.2.30.190:43609)
0808 10:09:23.363593 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 10:09:23.363597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:09:23.365510 (+ 1913us) server_negotiation.cc:407] Connection header received
0808 10:09:23.365549 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:09:23.365551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:09:23.365565 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:09:23.365651 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:09:23.369589 (+ 3938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.370401 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.372390 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.372701 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.372793 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:23.375554 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:09:23.375556 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:09:23.375556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:09:23.375567 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:09:23.377654 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:09:23.378171 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:09:23.378173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:09:23.378173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:09:23.378236 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:09:23.378420 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:09:23.378421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:09:23.378421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:09:23.378557 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:09:23.378639 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:09:23.378874 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 10:09:23.378961 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:09:23.382041 13838 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:09:23.385085 13838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:09:23.395080 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:23.388747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:35329 (local address 127.0.0.1:36266)
0808 10:09:23.390063 (+ 1316us) negotiation.cc:107] Waiting for socket to connect
0808 10:09:23.390073 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:09:23.390135 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:09:23.390432 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:09:23.390435 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:09:23.390438 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:09:23.390520 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:09:23.390522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.392033 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.392034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:09:23.393554 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:09:23.393558 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.394710 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.394711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:09:23.394804 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:23.394819 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 10:09:23.394898 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 10:09:23.394957 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1299,"mutex_wait_us":3,"thread_start_us":1253,"threads_started":1}
I20260808 10:09:23.395413 14362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:23.389171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36266 (local address 127.2.30.131:35329)
0808 10:09:23.389579 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0808 10:09:23.389584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:09:23.390134 (+ 550us) server_negotiation.cc:407] Connection header received
0808 10:09:23.390239 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:09:23.390242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:09:23.390260 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:09:23.390352 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:09:23.390605 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.391907 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.393691 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.394519 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.394595 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:23.394671 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:09:23.395218 (+ 547us) server_negotiation.cc:299] Negotiation successful
0808 10:09:23.395314 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":202,"threads_started":1}
I20260808 10:09:23.399003 14277 tablet_service.cc:1467] Processing CreateTablet for tablet 557aaae865594486b0f9180fb8dbb235 (DEFAULT_TABLE table=test-table [id=4da330ba94b74d1db345d6cc6519b979]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:09:23.399480 14277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 557aaae865594486b0f9180fb8dbb235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:09:23.407058 14363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:23.388747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:37303 (local address 127.0.0.1:48766)
0808 10:09:23.389850 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0808 10:09:23.389900 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 10:09:23.389980 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:09:23.391404 (+ 1424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:09:23.391406 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:09:23.391408 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:09:23.391459 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:09:23.391460 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.393340 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.393341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:09:23.405148 (+ 11807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:09:23.405155 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.406704 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.406705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:09:23.406804 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:23.406809 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:09:23.406862 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 10:09:23.406928 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1041,"thread_start_us":107,"threads_started":1}
I20260808 10:09:23.407437 14360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:23.389130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.2.30.129:37303)
0808 10:09:23.391160 (+ 2030us) server_negotiation.cc:206] Beginning negotiation
0808 10:09:23.391165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:09:23.391177 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:09:23.391214 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:09:23.391216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:09:23.391234 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:09:23.391319 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:09:23.391537 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.393227 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.405620 (+ 12393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.406504 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.406590 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:23.406659 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:09:23.407288 (+ 629us) server_negotiation.cc:299] Negotiation successful
0808 10:09:23.407343 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1985,"thread_start_us":74,"threads_started":1}
I20260808 10:09:23.410566 13991 tablet_service.cc:1467] Processing CreateTablet for tablet 557aaae865594486b0f9180fb8dbb235 (DEFAULT_TABLE table=test-table [id=4da330ba94b74d1db345d6cc6519b979]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:09:23.411043 13991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 557aaae865594486b0f9180fb8dbb235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:09:23.412163 14365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:23.388756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:35653 (local address 127.0.0.1:54804)
0808 10:09:23.394481 (+ 5725us) negotiation.cc:107] Waiting for socket to connect
0808 10:09:23.394495 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:09:23.404492 (+ 9997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:09:23.406162 (+ 1670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:09:23.406164 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:09:23.406168 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:09:23.406224 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:09:23.406226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.409293 (+ 3067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.409294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:09:23.410531 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:09:23.410536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.411895 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.411897 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:09:23.411975 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:23.411983 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:09:23.412018 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 10:09:23.412073 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5702,"thread_start_us":952,"threads_started":1}
I20260808 10:09:23.412640 14361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:23.389041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54804 (local address 127.2.30.130:35653)
0808 10:09:23.405863 (+ 16822us) server_negotiation.cc:206] Beginning negotiation
0808 10:09:23.405868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:09:23.405883 (+ 15us) server_negotiation.cc:407] Connection header received
0808 10:09:23.405936 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:09:23.405938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:09:23.405957 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:09:23.406057 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:09:23.407500 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.408849 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.410808 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.411708 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.412366 (+ 658us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:23.412477 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:09:23.412508 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 10:09:23.412555 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16751,"thread_start_us":210,"threads_started":1}
I20260808 10:09:23.413410 14135 tablet_service.cc:1467] Processing CreateTablet for tablet 557aaae865594486b0f9180fb8dbb235 (DEFAULT_TABLE table=test-table [id=4da330ba94b74d1db345d6cc6519b979]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:09:23.413775 14135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 557aaae865594486b0f9180fb8dbb235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:09:23.415643 14367 tablet_bootstrap.cc:492] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d: Bootstrap starting.
I20260808 10:09:23.416553 14367 tablet_bootstrap.cc:654] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d: Neither blocks nor log segments found. Creating new log.
I20260808 10:09:23.416980 14367 log.cc:824] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d: Log is configured to *not* fsync() on all Append() calls
I20260808 10:09:23.417979 14367 tablet_bootstrap.cc:492] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d: No bootstrap required, opened a new log
I20260808 10:09:23.418058 14367 ts_tablet_manager.cc:1397] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:09:23.423300 14366 tablet_bootstrap.cc:492] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b: Bootstrap starting.
I20260808 10:09:23.423360 14367 raft_consensus.cc:348] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558b5ef6c9cf49bb939a26c3f5ec482b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35329 } } peers { permanent_uuid: "ebe6e071121c47b9b2aaa4a98751a394" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35653 } } peers { permanent_uuid: "9394be1c3c48471d86d9416ffc6c9b6d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 37303 } }
I20260808 10:09:23.423815 14367 raft_consensus.cc:374] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:09:23.423906 14367 raft_consensus.cc:729] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9394be1c3c48471d86d9416ffc6c9b6d, State: Initialized, Role: FOLLOWER
I20260808 10:09:23.424122 14367 consensus_queue.cc:260] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558b5ef6c9cf49bb939a26c3f5ec482b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35329 } } peers { permanent_uuid: "ebe6e071121c47b9b2aaa4a98751a394" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35653 } } peers { permanent_uuid: "9394be1c3c48471d86d9416ffc6c9b6d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 37303 } }
I20260808 10:09:23.424656 14367 ts_tablet_manager.cc:1428] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d: Time spent starting tablet: real 0.007s user 0.000s sys 0.005s
I20260808 10:09:23.424856 14061 heartbeater.cc:503] Master 127.2.30.190:43609 was elected leader, sending a full tablet report...
I20260808 10:09:23.425428 14366 tablet_bootstrap.cc:654] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b: Neither blocks nor log segments found. Creating new log.
I20260808 10:09:23.426061 14369 tablet_bootstrap.cc:492] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394: Bootstrap starting.
I20260808 10:09:23.426254 14366 log.cc:824] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:09:23.428591 14369 tablet_bootstrap.cc:654] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394: Neither blocks nor log segments found. Creating new log.
I20260808 10:09:23.429632 14369 log.cc:824] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394: Log is configured to *not* fsync() on all Append() calls
I20260808 10:09:23.431543 14369 tablet_bootstrap.cc:492] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394: No bootstrap required, opened a new log
I20260808 10:09:23.431687 14369 ts_tablet_manager.cc:1397] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 10:09:23.435668 14369 raft_consensus.cc:348] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558b5ef6c9cf49bb939a26c3f5ec482b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35329 } } peers { permanent_uuid: "ebe6e071121c47b9b2aaa4a98751a394" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35653 } } peers { permanent_uuid: "9394be1c3c48471d86d9416ffc6c9b6d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 37303 } }
I20260808 10:09:23.435979 14369 raft_consensus.cc:374] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:09:23.436084 14369 raft_consensus.cc:729] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebe6e071121c47b9b2aaa4a98751a394, State: Initialized, Role: FOLLOWER
I20260808 10:09:23.436422 14369 consensus_queue.cc:260] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558b5ef6c9cf49bb939a26c3f5ec482b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35329 } } peers { permanent_uuid: "ebe6e071121c47b9b2aaa4a98751a394" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35653 } } peers { permanent_uuid: "9394be1c3c48471d86d9416ffc6c9b6d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 37303 } }
I20260808 10:09:23.436803 14369 ts_tablet_manager.cc:1428] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 10:09:23.436986 14205 heartbeater.cc:503] Master 127.2.30.190:43609 was elected leader, sending a full tablet report...
I20260808 10:09:23.438190 14366 tablet_bootstrap.cc:492] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b: No bootstrap required, opened a new log
I20260808 10:09:23.438308 14366 ts_tablet_manager.cc:1397] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20260808 10:09:23.442459 14366 raft_consensus.cc:348] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558b5ef6c9cf49bb939a26c3f5ec482b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35329 } } peers { permanent_uuid: "ebe6e071121c47b9b2aaa4a98751a394" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35653 } } peers { permanent_uuid: "9394be1c3c48471d86d9416ffc6c9b6d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 37303 } }
I20260808 10:09:23.443570 14370 raft_consensus.cc:482] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:09:23.443816 14370 raft_consensus.cc:504] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558b5ef6c9cf49bb939a26c3f5ec482b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35329 } } peers { permanent_uuid: "ebe6e071121c47b9b2aaa4a98751a394" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35653 } } peers { permanent_uuid: "9394be1c3c48471d86d9416ffc6c9b6d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 37303 } }
I20260808 10:09:23.444358 14370 leader_election.cc:288] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 558b5ef6c9cf49bb939a26c3f5ec482b (127.2.30.131:35329), ebe6e071121c47b9b2aaa4a98751a394 (127.2.30.130:35653)
I20260808 10:09:23.445895 14366 raft_consensus.cc:374] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:09:23.446089 14366 raft_consensus.cc:729] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 558b5ef6c9cf49bb939a26c3f5ec482b, State: Initialized, Role: FOLLOWER
I20260808 10:09:23.446363 14366 consensus_queue.cc:260] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558b5ef6c9cf49bb939a26c3f5ec482b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35329 } } peers { permanent_uuid: "ebe6e071121c47b9b2aaa4a98751a394" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35653 } } peers { permanent_uuid: "9394be1c3c48471d86d9416ffc6c9b6d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 37303 } }
I20260808 10:09:23.446853 14366 ts_tablet_manager.cc:1428] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260808 10:09:23.447065 14347 heartbeater.cc:503] Master 127.2.30.190:43609 was elected leader, sending a full tablet report...
I20260808 10:09:23.456413 14361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:23.445050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:59113 (local address 127.2.30.130:35653)
0808 10:09:23.445117 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 10:09:23.445120 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:09:23.448292 (+ 3172us) server_negotiation.cc:407] Connection header received
0808 10:09:23.448415 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:09:23.448416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:09:23.448430 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:09:23.448462 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:09:23.449488 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.450301 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.454361 (+ 4060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.454931 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.455002 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:23.455014 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 10:09:23.456235 (+ 1221us) server_negotiation.cc:299] Negotiation successful
0808 10:09:23.456326 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 10:09:23.457008 14374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:23.444660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:35653 (local address 127.2.30.129:59113)
0808 10:09:23.448190 (+ 3530us) negotiation.cc:107] Waiting for socket to connect
0808 10:09:23.448204 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:09:23.448330 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:09:23.448539 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:09:23.448540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:09:23.448542 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:09:23.448748 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:09:23.448751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.450476 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.450477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:09:23.451846 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:09:23.451849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.455082 (+ 3233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.455083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:09:23.455185 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:23.455190 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:09:23.456781 (+ 1591us) client_negotiation.cc:240] Negotiation successful
0808 10:09:23.456838 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3443,"thread_start_us":183,"threads_started":1}
I20260808 10:09:23.457441 14362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:23.445729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:45795 (local address 127.2.30.131:35329)
0808 10:09:23.445776 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:09:23.445778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:09:23.448971 (+ 3193us) server_negotiation.cc:407] Connection header received
0808 10:09:23.449078 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:09:23.449079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:09:23.449091 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:09:23.449120 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:09:23.451983 (+ 2863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.452754 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.455302 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.455878 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.455938 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:23.455948 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:09:23.457300 (+ 1352us) server_negotiation.cc:299] Negotiation successful
0808 10:09:23.457372 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 10:09:23.458354 14156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "557aaae865594486b0f9180fb8dbb235" candidate_uuid: "9394be1c3c48471d86d9416ffc6c9b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebe6e071121c47b9b2aaa4a98751a394" is_pre_election: true
I20260808 10:09:23.458534 14156 raft_consensus.cc:2454] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9394be1c3c48471d86d9416ffc6c9b6d in term 0.
I20260808 10:09:23.458951 13942 leader_election.cc:302] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9394be1c3c48471d86d9416ffc6c9b6d, ebe6e071121c47b9b2aaa4a98751a394; no voters:
I20260808 10:09:23.459139 14370 raft_consensus.cc:2781] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:09:23.459190 14370 raft_consensus.cc:482] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:09:23.459213 14370 raft_consensus.cc:3037] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:09:23.460567 14370 raft_consensus.cc:504] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558b5ef6c9cf49bb939a26c3f5ec482b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35329 } } peers { permanent_uuid: "ebe6e071121c47b9b2aaa4a98751a394" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35653 } } peers { permanent_uuid: "9394be1c3c48471d86d9416ffc6c9b6d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 37303 } }
I20260808 10:09:23.460721 14370 leader_election.cc:288] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [CANDIDATE]: Term 1 election: Requested vote from peers 558b5ef6c9cf49bb939a26c3f5ec482b (127.2.30.131:35329), ebe6e071121c47b9b2aaa4a98751a394 (127.2.30.130:35653)
I20260808 10:09:23.461118 14156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "557aaae865594486b0f9180fb8dbb235" candidate_uuid: "9394be1c3c48471d86d9416ffc6c9b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebe6e071121c47b9b2aaa4a98751a394"
I20260808 10:09:23.461210 14156 raft_consensus.cc:3037] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:09:23.462891 14156 raft_consensus.cc:2454] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9394be1c3c48471d86d9416ffc6c9b6d in term 1.
I20260808 10:09:23.463220 13942 leader_election.cc:302] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9394be1c3c48471d86d9416ffc6c9b6d, ebe6e071121c47b9b2aaa4a98751a394; no voters:
I20260808 10:09:23.463377 14370 raft_consensus.cc:2781] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:09:23.463553 14370 raft_consensus.cc:686] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [term 1 LEADER]: Becoming Leader. State: Replica: 9394be1c3c48471d86d9416ffc6c9b6d, State: Running, Role: LEADER
I20260808 10:09:23.463660 14370 consensus_queue.cc:237] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558b5ef6c9cf49bb939a26c3f5ec482b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35329 } } peers { permanent_uuid: "ebe6e071121c47b9b2aaa4a98751a394" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35653 } } peers { permanent_uuid: "9394be1c3c48471d86d9416ffc6c9b6d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 37303 } }
I20260808 10:09:23.464591 13838 catalog_manager.cc:5495] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9394be1c3c48471d86d9416ffc6c9b6d (127.2.30.129). New cstate: current_term: 1 leader_uuid: "9394be1c3c48471d86d9416ffc6c9b6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "558b5ef6c9cf49bb939a26c3f5ec482b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebe6e071121c47b9b2aaa4a98751a394" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9394be1c3c48471d86d9416ffc6c9b6d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 37303 } health_report { overall_health: HEALTHY } } }
I20260808 10:09:23.457928 14375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:23.445364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:35329 (local address 127.2.30.129:45795)
0808 10:09:23.448908 (+ 3544us) negotiation.cc:107] Waiting for socket to connect
0808 10:09:23.448919 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:09:23.448998 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:09:23.449185 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:09:23.449186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:09:23.449188 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:09:23.449389 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:09:23.449391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.452924 (+ 3533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.452925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:09:23.454224 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:09:23.454228 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.456024 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.456025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:09:23.456132 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:23.456136 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:09:23.457692 (+ 1556us) client_negotiation.cc:240] Negotiation successful
0808 10:09:23.457748 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3497,"thread_start_us":167,"threads_started":1}
I20260808 10:09:23.466413 14298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "557aaae865594486b0f9180fb8dbb235" candidate_uuid: "9394be1c3c48471d86d9416ffc6c9b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558b5ef6c9cf49bb939a26c3f5ec482b"
I20260808 10:09:23.466562 14298 raft_consensus.cc:3037] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:09:23.468475 14298 raft_consensus.cc:2454] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9394be1c3c48471d86d9416ffc6c9b6d in term 1.
I20260808 10:09:23.468918 14299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "557aaae865594486b0f9180fb8dbb235" candidate_uuid: "9394be1c3c48471d86d9416ffc6c9b6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "558b5ef6c9cf49bb939a26c3f5ec482b" is_pre_election: true
I20260808 10:09:23.469022 14299 raft_consensus.cc:2362] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9394be1c3c48471d86d9416ffc6c9b6d in term 1. Re-sending same reply.
I20260808 10:09:23.478880 14360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:23.475118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.2.30.129:37303)
0808 10:09:23.475167 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 10:09:23.475169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:09:23.475298 (+ 129us) server_negotiation.cc:407] Connection header received
0808 10:09:23.475364 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:09:23.475365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:09:23.475391 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:09:23.475459 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:09:23.475671 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.476407 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.477527 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:09:23.477814 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:09:23.477884 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:09:23.478123 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:09:23.478287 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:09:23.478376 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:09:23.478627 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 10:09:23.478654 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 10:09:23.481029 14206 tablet.cc:2369] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:09:23.482803 14156 raft_consensus.cc:1261] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394 [term 1 FOLLOWER]: Refusing update from remote peer 9394be1c3c48471d86d9416ffc6c9b6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:09:23.483430 14377 consensus_queue.cc:1035] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebe6e071121c47b9b2aaa4a98751a394" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:09:23.486316 14299 raft_consensus.cc:1261] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b [term 1 FOLLOWER]: Refusing update from remote peer 9394be1c3c48471d86d9416ffc6c9b6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:09:23.486703 14377 consensus_queue.cc:1035] T 557aaae865594486b0f9180fb8dbb235 P 9394be1c3c48471d86d9416ffc6c9b6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "558b5ef6c9cf49bb939a26c3f5ec482b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:09:23.493145 14380 mvcc.cc:205] Tried to move back new op lower bound from 7316208695221579776 to 7316208695147790336. Current Snapshot: MvccSnapshot[applied={T|T < 7316208695221579776}]
I20260808 10:09:23.493489 14383 mvcc.cc:205] Tried to move back new op lower bound from 7316208695221579776 to 7316208695147790336. Current Snapshot: MvccSnapshot[applied={T|T < 7316208695221579776}]
I20260808 10:09:23.496268 14381 mvcc.cc:205] Tried to move back new op lower bound from 7316208695221579776 to 7316208695147790336. Current Snapshot: MvccSnapshot[applied={T|T < 7316208695221579776}]
I20260808 10:09:23.502009 13838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:09:23.502143 13838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:09:23.503486 13838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7f00d5a240e14359a85cf9a5b06ce63f: Sending DeleteTablet for 3 replicas of tablet 557aaae865594486b0f9180fb8dbb235
I20260808 10:09:23.504280 14277 tablet_service.cc:1514] Processing DeleteTablet for tablet 557aaae865594486b0f9180fb8dbb235 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:09:23 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:36266
I20260808 10:09:23.504623 14135 tablet_service.cc:1514] Processing DeleteTablet for tablet 557aaae865594486b0f9180fb8dbb235 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:09:23 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:54804
I20260808 10:09:23.504889 13991 tablet_service.cc:1514] Processing DeleteTablet for tablet 557aaae865594486b0f9180fb8dbb235 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:09:23 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:48766
I20260808 10:09:23.505861 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 13924
I20260808 10:09:23.511211 14389 tablet_replica.cc:330] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b: stopping tablet replica
I20260808 10:09:23.511471 14389 raft_consensus.cc:2229] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:09:23.511715 14389 raft_consensus.cc:2258] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:09:23.512360 14389 ts_tablet_manager.cc:1905] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:09:23.512560 14390 tablet_replica.cc:330] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394: stopping tablet replica
I20260808 10:09:23.512720 14390 raft_consensus.cc:2229] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:09:23.512888 14390 raft_consensus.cc:2258] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:09:23.515560 14390 ts_tablet_manager.cc:1905] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:09:23.517424 14389 ts_tablet_manager.cc:1918] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:09:23.517597 14389 log.cc:1196] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/557aaae865594486b0f9180fb8dbb235
I20260808 10:09:23.518374 14389 ts_tablet_manager.cc:1939] T 557aaae865594486b0f9180fb8dbb235 P 558b5ef6c9cf49bb939a26c3f5ec482b: Deleting consensus metadata
I20260808 10:09:23.518581 14390 ts_tablet_manager.cc:1918] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:09:23.518738 14390 log.cc:1196] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/557aaae865594486b0f9180fb8dbb235
I20260808 10:09:23.519111 14390 ts_tablet_manager.cc:1939] T 557aaae865594486b0f9180fb8dbb235 P ebe6e071121c47b9b2aaa4a98751a394: Deleting consensus metadata
I20260808 10:09:23.519724 13823 catalog_manager.cc:4841] TS ebe6e071121c47b9b2aaa4a98751a394 (127.2.30.130:35653): tablet 557aaae865594486b0f9180fb8dbb235 (table test-table [id=4da330ba94b74d1db345d6cc6519b979]) successfully deleted
I20260808 10:09:23.520407 13825 catalog_manager.cc:4841] TS 558b5ef6c9cf49bb939a26c3f5ec482b (127.2.30.131:35329): tablet 557aaae865594486b0f9180fb8dbb235 (table test-table [id=4da330ba94b74d1db345d6cc6519b979]) successfully deleted
W20260808 10:09:23.523336 13824 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:37303 (error 108)
I20260808 10:09:23.523568 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 14068
I20260808 10:09:23.526170 14365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:09:23.526015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:37303 (local address 127.0.0.1:48778)
0808 10:09:23.526070 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 10:09:23.526118 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:37303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 10:09:23.526486 13824 catalog_manager.cc:4584] TS 9394be1c3c48471d86d9416ffc6c9b6d (127.2.30.129:37303): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 557aaae865594486b0f9180fb8dbb235: Network error: Client connection negotiation failed: client connection to 127.2.30.129:37303: connect: Connection refused (error 111)
I20260808 10:09:23.542171 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 14212
I20260808 10:09:23.552913 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 13805
I20260808 10:09:23.560916 2170 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 10:09:23.917418 2170 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 10:09:23.917593 2170 mini_ranger.cc:66] Stopping Ranger...
I20260808 10:09:24.273592 2170 mini_ranger.cc:68] Stopped Ranger
2026-08-08 10:09:24.273 UTC [13368] LOG: received smart shutdown request
2026-08-08 10:09:24.279 UTC [13368] LOG: background worker "logical replication launcher" (PID 13378) exited with exit code 1
2026-08-08 10:09:24.280 UTC [13373] LOG: shutting down
2026-08-08 10:09:24.509 UTC [13368] LOG: database system is shut down
2026-08-08T10:09:24Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53845 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:09:24 dist-test-slave-w206 krb5kdc[14401](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:09:24 dist-test-slave-w206 krb5kdc[14401](info): set up 2 sockets
Aug 08 10:09:24 dist-test-slave-w206 krb5kdc[14401](info): commencing operation
krb5kdc: starting...
W20260808 10:09:25.791707 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:09:25 dist-test-slave-w206 krb5kdc[14401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:09:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:09:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.30.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.30.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.30.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/rangeradmin_127.2.30.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.30.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/rangeradmin_127.2.30.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.30.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.30.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.30.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/rangerlookup_127.2.30.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.30.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/rangerlookup_127.2.30.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.30.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.30.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.30.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/HTTP_127.2.30.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.30.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/HTTP_127.2.30.148@KRBTEST.COM.keytab.
I20260808 10:09:25.994587 2170 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskbFyXgX/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/postgres -l logfile start
2026-08-08 10:09:28.904 UTC [14431] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 10:09:28.904 UTC [14431] LOG: listening on IPv4 address "127.2.30.148", port 57927
2026-08-08 10:09:28.908 UTC [14431] LOG: listening on Unix socket "/tmp/.s.PGSQL.57927"
2026-08-08 10:09:28.939 UTC [14434] LOG: database system was shut down at 2026-08-08 10:09:27 UTC
2026-08-08 10:09:28.943 UTC [14431] LOG: database system is ready to accept connections
I20260808 10:09:29.935523 2170 mini_postgres.cc:96] Postgres bound to 57927
2026-08-08 10:09:29.943 UTC [14445] FATAL: database "slave" does not exist
127.2.30.148:57927 - accepting connections
I20260808 10:09:29.943789 2170 mini_ranger.cc:161] Starting Ranger...
I20260808 10:09:29.960904 2170 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 10:09:30.530345 2170 mini_ranger.cc:87] Created ranger Postgres database
I20260808 10:09:30.530447 2170 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-admin
2026-08-08 10:09:30,815 [I] DB FLAVOR :POSTGRES
2026-08-08 10:09:30,816 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:09:30,816 [I] Checking connection..
2026-08-08 10:09:30,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 10:09:31,071 [I] Checking connection passed.
2026-08-08 10:09:31,071 [I] --------- Verifying version history table ---------
2026-08-08 10:09:31,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:09:31,326 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:09:31,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:09:31,595 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 10:09:31,595 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 10:09:31,596 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 10:09:31.877 UTC [14512] WARNING: there is no transaction in progress
2026-08-08 10:09:31,885 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 10:09:31,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 10:09:32,146 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 10:09:32,146 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:09:32,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 10:09:32,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 10:09:32,712 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-w206.c.gcp-upstream.internal','N') ;"
2026-08-08 10:09:32,980 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:09:33,245 [I] Table x_portal_user does not exist in database ranger
2026-08-08 10:09:33,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:09:33,545 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 10:09:33,546 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:09:33,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:09:34,074 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 10:09:34,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 10:09:35.175 UTC [14624] WARNING: there is no transaction in progress
2026-08-08 10:09:35.195 UTC [14624] WARNING: there is no transaction in progress
2026-08-08 10:09:35.215 UTC [14624] WARNING: there is no transaction in progress
2026-08-08 10:09:35.234 UTC [14624] WARNING: there is no transaction in progress
2026-08-08 10:09:35.255 UTC [14624] WARNING: there is no transaction in progress
2026-08-08 10:09:35.274 UTC [14624] WARNING: there is no transaction in progress
2026-08-08 10:09:35.338 UTC [14624] WARNING: there is no transaction in progress
2026-08-08 10:09:35.346 UTC [14624] WARNING: there is no transaction in progress
2026-08-08 10:09:35.362 UTC [14624] WARNING: there is no transaction in progress
2026-08-08 10:09:35.375 UTC [14624] WARNING: there is no transaction in progress
2026-08-08 10:09:35.387 UTC [14624] WARNING: there is no transaction in progress
2026-08-08 10:09:35.405 UTC [14624] WARNING: there is no transaction in progress
2026-08-08 10:09:35.419 UTC [14624] WARNING: there is no transaction in progress
2026-08-08 10:09:35.427 UTC [14624] WARNING: there is no transaction in progress
2026-08-08 10:09:35.436 UTC [14624] WARNING: there is no transaction in progress
2026-08-08 10:09:35.445 UTC [14624] WARNING: there is no transaction in progress
2026-08-08 10:09:36,217 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 10:09:36,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 10:09:36,471 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 10:09:36,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 10:09:36,742 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 10:09:36,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 10:09:37,030 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 10:09:37,030 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 10:09:37,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 10:09:37,562 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-w206.c.gcp-upstream.internal';"
2026-08-08 10:09:37,840 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 10:09:37,840 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 10:09:38,109 [I] DB_PATCHES have already been applied
I20260808 10:09:38.116257 2170 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-admin:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskbFyXgX/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 10:09:38.116395 2170 mini_ranger.cc:193] Using host: 127.2.30.148
I20260808 10:09:38.120739 2170 mini_ranger.cc:227] Ranger admin URL: http://127.2.30.148:56139
Aug 08, 2026 10:09:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:09:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:09:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 10:09:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 10:09:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 10:09:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 10:09:39 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 10:09:39.302459 2170 mini_ranger.cc:160] Time spent starting Ranger: real 9.359s user 0.000s sys 0.012s
Aug 08, 2026 10:10:02 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 10:10:02 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 10:10:02 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 10:10:02 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 10:10:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 10:10:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 10:10:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 10:10:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 10:10:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 10:10:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 10:10:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 10:10:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 10:10:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 10:10:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 10:10:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 10:10:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 10:10:02 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 10:10:03 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 10:10:04.109129 2170 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.30.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.30.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.30.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/rangerkms_127.2.30.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.30.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/rangerkms_127.2.30.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.30.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.30.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.30.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/HTTP_127.2.30.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.30.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/HTTP_127.2.30.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 10:10:04 dist-test-slave-w206 krb5kdc[14401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183804, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 10:10:04.230712 2170 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 10:10:04.250380 2170 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 10:10:05.343396 2170 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 10:10:05.343575 2170 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-kms
I20260808 10:10:05.343639 2170 mini_ranger_kms.cc:227] Using postgres at 127.2.30.148:57927
2026-08-08 10:10:05,532 [I] DB FLAVOR :POSTGRES
2026-08-08 10:10:05,533 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 10:10:05,533 [I] Checking connection
2026-08-08 10:10:05,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 10:10:05,797 [I] connection success
2026-08-08 10:10:05,798 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 10:10:05,798 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 10:10:05,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 10:10:06,073 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 10:10:06,074 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 10:10:06,074 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 10:10:06,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.30.148:57927/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 10:10:06,363 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 10:10:06.922854 2170 mini_ranger_kms.cc:310] Created kms service
I20260808 10:10:07.185904 2170 mini_ranger_kms.cc:326] Created kudu user
I20260808 10:10:07.244079 2170 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 10:10:07 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183804, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
I20260808 10:10:07.719933 2170 mini_ranger_kms.cc:384] Added ranger policy
I20260808 10:10:07.720182 2170 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-kms:/tmp/dist-test-taskbFyXgX/build/release/bin/postgresql.jar:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskbFyXgX/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 10:10:07.720216 2170 mini_ranger_kms.cc:240] Using host: 127.2.30.148
I20260808 10:10:07.725224 2170 mini_ranger_kms.cc:276] Ranger KMS PID: 14816
I20260808 10:10:07.725330 2170 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.30.148:46191
Aug 08, 2026 10:10:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:10:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:10:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 10:10:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 10:10:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 10:10:09 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 10:10:10 dist-test-slave-w206 krb5kdc[14401](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786183810, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.30.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:10.093735 2170 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.863s user 0.006s sys 0.169s
I20260808 10:10:10.093842 2170 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 10:10:10.093894 2170 mini_ranger_kms.cc:401] 127.2.30.148:46191/kms/v1/keys
Aug 08 10:10:10 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183810, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.30.148@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
Aug 08 10:10:10 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183810, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.30.148@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
Aug 08 10:10:13 dist-test-slave-w206 krb5kdc[14401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:10:13.346544 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:36937
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:36937
--ranger_config_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.30.148:46191/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:13.450219 14868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:13.450465 14868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:13.450491 14868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:13.452149 14868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:10:13.452209 14868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:13.452224 14868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:13.452237 14868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:10:13.452342 14868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:10:13.454357 14868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.30.148:46191/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:36937
--ranger_config_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:36937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:13.454668 14868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:13.454955 14868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:13.461813 14874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:13.461861 14868 server_base.cc:1034] running on GCE node
W20260808 10:10:13.461956 14876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:13.461824 14873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:13.462399 14868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:13.462772 14868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:13.463970 14868 hybrid_clock.cc:648] HybridClock initialized: now 1786183813463929 us; error 56 us; skew 500 ppm
Aug 08 10:10:13 dist-test-slave-w206 krb5kdc[14401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183813, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:13.472303 14868 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:10:13.472878 14868 webserver.cc:460] Webserver started at http://127.2.30.190:43037/ using document root <none> and password file <none>
I20260808 10:10:13.473169 14868 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:13.473214 14868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:13.473376 14868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:10:13 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183813, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
I20260808 10:10:13.545244 14868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "18f937798818495cafadca0ccee59eb5"
format_stamp: "Formatted at 2026-08-08 10:10:13 on dist-test-slave-w206"
server_key: "65c2d823972c848fd98d57059f42b3ce"
server_key_iv: "920a533f979446651359a52c843e2ac5"
server_key_version: "kuduclusterkey@0"
I20260808 10:10:13.545792 14868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "18f937798818495cafadca0ccee59eb5"
format_stamp: "Formatted at 2026-08-08 10:10:13 on dist-test-slave-w206"
server_key: "65c2d823972c848fd98d57059f42b3ce"
server_key_iv: "920a533f979446651359a52c843e2ac5"
server_key_version: "kuduclusterkey@0"
I20260808 10:10:13.547555 14868 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 10:10:13.572005 14883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:13.572295 14868 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:13.572422 14868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "18f937798818495cafadca0ccee59eb5"
format_stamp: "Formatted at 2026-08-08 10:10:13 on dist-test-slave-w206"
server_key: "65c2d823972c848fd98d57059f42b3ce"
server_key_iv: "920a533f979446651359a52c843e2ac5"
server_key_version: "kuduclusterkey@0"
I20260808 10:10:13.572530 14868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:13.592438 14868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:13.595593 14868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:13.595798 14868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:13.602228 14868 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:36937
I20260808 10:10:13.602310 14939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:36937 every 8 connection(s)
I20260808 10:10:13.602738 14868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:10:13.603554 14940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:13.606174 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 14868
I20260808 10:10:13.606302 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:10:13.606298 14940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5: Bootstrap starting.
I20260808 10:10:13.606496 2170 external_mini_cluster.cc:1428] Setting key 4fe8f209bd06aea5f3a77d2fb56899e4
I20260808 10:10:13.607252 14940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:13.607781 14940 log.cc:824] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:13.608709 14940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5: No bootstrap required, opened a new log
I20260808 10:10:13.610603 14940 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f937798818495cafadca0ccee59eb5" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36937 } }
I20260808 10:10:13.610809 14940 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:13.610857 14940 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f937798818495cafadca0ccee59eb5, State: Initialized, Role: FOLLOWER
I20260808 10:10:13.611078 14940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f937798818495cafadca0ccee59eb5" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36937 } }
I20260808 10:10:13.611188 14940 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:10:13.611225 14940 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:10:13.611305 14940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:13.612226 14940 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f937798818495cafadca0ccee59eb5" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36937 } }
I20260808 10:10:13.612370 14940 leader_election.cc:302] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18f937798818495cafadca0ccee59eb5; no voters:
I20260808 10:10:13.612577 14940 leader_election.cc:288] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:10:13.612772 14946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:10:13.612946 14940 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:10:13.612982 14946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 1 LEADER]: Becoming Leader. State: Replica: 18f937798818495cafadca0ccee59eb5, State: Running, Role: LEADER
I20260808 10:10:13.613158 14946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f937798818495cafadca0ccee59eb5" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36937 } }
I20260808 10:10:13.613574 14946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18f937798818495cafadca0ccee59eb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f937798818495cafadca0ccee59eb5" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36937 } } }
I20260808 10:10:13.613765 14946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [sys.catalog]: This master's current role is: LEADER
I20260808 10:10:13.613978 14946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18f937798818495cafadca0ccee59eb5. Latest consensus state: current_term: 1 leader_uuid: "18f937798818495cafadca0ccee59eb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f937798818495cafadca0ccee59eb5" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36937 } } }
I20260808 10:10:13.614108 14946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [sys.catalog]: This master's current role is: LEADER
I20260808 10:10:13.614782 14940 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 10:10:13.615602 14955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 10:10:13 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:13.619943 14955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:10:13.625166 14943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:13.607205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35756 (local address 127.2.30.190:36937)
0808 10:10:13.608100 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:13.608108 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:13.608137 (+ 29us) server_negotiation.cc:407] Connection header received
0808 10:10:13.608764 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:13.608773 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:13.608847 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:13.609020 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:13.609879 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:13.611784 (+ 1905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:13.612980 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:13.613644 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:13.613730 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:13.618112 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:13.618157 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:13.618157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:13.618256 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:13.621825 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:13.622344 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:13.622348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:13.622348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:13.622479 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:13.622703 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:13.622704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:13.622704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:13.623522 (+ 818us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:13.624399 (+ 877us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:13.624483 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 10:10:13.624841 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":848,"thread_start_us":91,"threads_started":1}
I20260808 10:10:13.627203 14955 catalog_manager.cc:1347] Generated new cluster ID: 9356baacc7214736b97dbf55b026f9bf
I20260808 10:10:13.627270 14955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:10:13.669011 14955 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:10:13.670009 14955 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:10:13.679742 14955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5: Generated new TSK 0
I20260808 10:10:13.679987 14955 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:10:14.440060 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.30.148:46191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36937
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:14.572108 14988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:14.572443 14988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:14.572489 14988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:14.574435 14988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:14.574512 14988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:14.574555 14988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:10:14.577028 14988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.30.148:46191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:14.577358 14988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:14.577674 14988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:14.584746 14994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:14.584883 14993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:14.584781 14996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:14.585081 14988 server_base.cc:1034] running on GCE node
I20260808 10:10:14.585304 14988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:14.585687 14988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:14.586865 14988 hybrid_clock.cc:648] HybridClock initialized: now 1786183814586824 us; error 51 us; skew 500 ppm
Aug 08 10:10:14 dist-test-slave-w206 krb5kdc[14401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183814, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:14.590440 14988 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:10:14.591176 14988 webserver.cc:460] Webserver started at http://127.2.30.129:41589/ using document root <none> and password file <none>
I20260808 10:10:14.591460 14988 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:14.591536 14988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:14.591732 14988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:10:14 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183814, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
I20260808 10:10:14.606741 14988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "b11e6b67d1854791b8a394a096e2a3f3"
format_stamp: "Formatted at 2026-08-08 10:10:14 on dist-test-slave-w206"
server_key: "886b056c4be4ce01b446c05be1999276"
server_key_iv: "3ea2e562c5a5a040f8f2cd0e5914b813"
server_key_version: "kuduclusterkey@0"
I20260808 10:10:14.607316 14988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "b11e6b67d1854791b8a394a096e2a3f3"
format_stamp: "Formatted at 2026-08-08 10:10:14 on dist-test-slave-w206"
server_key: "886b056c4be4ce01b446c05be1999276"
server_key_iv: "3ea2e562c5a5a040f8f2cd0e5914b813"
server_key_version: "kuduclusterkey@0"
I20260808 10:10:14.609920 14988 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:10:14.624589 15006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:14.624871 14988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:14.624979 14988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "b11e6b67d1854791b8a394a096e2a3f3"
format_stamp: "Formatted at 2026-08-08 10:10:14 on dist-test-slave-w206"
server_key: "886b056c4be4ce01b446c05be1999276"
server_key_iv: "3ea2e562c5a5a040f8f2cd0e5914b813"
server_key_version: "kuduclusterkey@0"
I20260808 10:10:14.625102 14988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:14.646642 14988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:14.649353 14988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:14.649609 14988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:14.649953 14988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:14.650417 14988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:10:14.650521 14988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:14.650625 14988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:10:14.650708 14988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:14.657949 14988 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:38567
I20260808 10:10:14.658509 14988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:10:14.659698 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 14988
I20260808 10:10:14.659852 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:10:14.660109 2170 external_mini_cluster.cc:1428] Setting key a2412f4661cee42b9e6cea71cbb3b85c
I20260808 10:10:14.661408 15124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:38567 every 8 connection(s)
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Aug 08 10:10:14 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183814, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:14.682025 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:14.662006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:41017 (local address 127.2.30.190:36937)
0808 10:10:14.671138 (+ 9132us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:14.671141 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:14.671152 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:10:14.671185 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:14.671186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:14.671204 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:14.671298 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:14.672061 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:14.672719 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:14.673896 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:14.674134 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:14.675349 (+ 1215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:14.677931 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:14.677934 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:14.677934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:14.677944 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:14.680082 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:14.680653 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:14.680654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:14.680655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:14.680733 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:14.681101 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:14.681102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:14.681102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:14.681275 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:14.681354 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:14.681748 (+ 394us) server_negotiation.cc:299] Negotiation successful
0808 10:10:14.681861 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9090,"thread_start_us":94,"threads_started":1}
I20260808 10:10:14.682161 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:14.662003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36937 (local address 127.2.30.129:41017)
0808 10:10:14.668693 (+ 6690us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:14.668770 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:14.669698 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:14.671468 (+ 1770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:14.671475 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:14.671712 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:14.671940 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:14.671945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:14.672838 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:14.672839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:14.673780 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:14.673788 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:14.674863 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:14.674865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:14.675068 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:14.675754 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:14.675766 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:14.677794 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:14.680195 (+ 2401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:14.680204 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:14.680206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:14.680563 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:14.680841 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:14.680842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:14.680843 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:14.681020 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:14.681349 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:14.681358 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:14.681554 (+ 196us) client_negotiation.cc:769] Sending connection context
0808 10:10:14.681704 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 10:10:14.681894 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6622,"thread_start_us":87,"threads_started":1}
I20260808 10:10:14.683282 15125 heartbeater.cc:347] Connected to a master server at 127.2.30.190:36937
I20260808 10:10:14.683410 15125 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:14.683665 15125 heartbeater.cc:511] Master 127.2.30.190:36937 requested a full tablet report, sending...
I20260808 10:10:14.687533 14901 ts_manager.cc:194] Registered new tserver with Master: b11e6b67d1854791b8a394a096e2a3f3 (127.2.30.129:38567)
I20260808 10:10:14.688565 14901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:41017
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:10:14.698470 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.30.148:46191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36937
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:10:14 dist-test-slave-w206 krb5kdc[14401](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786183814, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 10:10:14.832645 15131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:14.833073 15131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:14.833165 15131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:14.835283 15131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:14.835371 15131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:14.835420 15131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:10:14.837972 15131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.30.148:46191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:14.838464 15131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:14.838860 15131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:14.847733 15141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:14.848832 15131 server_base.cc:1034] running on GCE node
W20260808 10:10:14.847733 15139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:14.849962 15138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:14.850431 15131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:14.850780 15131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:14.852144 15131 hybrid_clock.cc:648] HybridClock initialized: now 1786183814852108 us; error 52 us; skew 500 ppm
Aug 08 10:10:14 dist-test-slave-w206 krb5kdc[14401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183814, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:14.856240 15131 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:10:14.856894 15131 webserver.cc:460] Webserver started at http://127.2.30.130:44325/ using document root <none> and password file <none>
I20260808 10:10:14.857165 15131 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:14.857244 15131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:14.857398 15131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:10:14 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183814, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
I20260808 10:10:14.872172 15131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "fb7601ed0384490684d4be0e4b5c6535"
format_stamp: "Formatted at 2026-08-08 10:10:14 on dist-test-slave-w206"
server_key: "e69462754b0d0c6bd60631883d1416e7"
server_key_iv: "fd7a9118994c84c609cd3ffb35d83ab1"
server_key_version: "kuduclusterkey@0"
I20260808 10:10:14.872722 15131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "fb7601ed0384490684d4be0e4b5c6535"
format_stamp: "Formatted at 2026-08-08 10:10:14 on dist-test-slave-w206"
server_key: "e69462754b0d0c6bd60631883d1416e7"
server_key_iv: "fd7a9118994c84c609cd3ffb35d83ab1"
server_key_version: "kuduclusterkey@0"
I20260808 10:10:14.880579 15131 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.003s
I20260808 10:10:14.901014 15148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:14.901219 15131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:14.901315 15131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "fb7601ed0384490684d4be0e4b5c6535"
format_stamp: "Formatted at 2026-08-08 10:10:14 on dist-test-slave-w206"
server_key: "e69462754b0d0c6bd60631883d1416e7"
server_key_iv: "fd7a9118994c84c609cd3ffb35d83ab1"
server_key_version: "kuduclusterkey@0"
I20260808 10:10:14.901417 15131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:14.922509 15131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:14.925333 15131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:14.925508 15131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:14.925758 15131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:14.926208 15131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:10:14.926270 15131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:14.926311 15131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:10:14.926337 15131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:14.942160 15131 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:33417
I20260808 10:10:14.942691 15131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:10:14.944099 15265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:33417 every 8 connection(s)
I20260808 10:10:14.949062 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 15131
I20260808 10:10:14.949177 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:10:14.949380 2170 external_mini_cluster.cc:1428] Setting key ccbe485f61272641fc2c1ba2173e3ccd
Aug 08 10:10:14 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183814, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:14.958824 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:14.943802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:33539 (local address 127.2.30.190:36937)
0808 10:10:14.943858 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:14.943861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:14.945006 (+ 1145us) server_negotiation.cc:407] Connection header received
0808 10:10:14.946860 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:14.946862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:14.946886 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:14.946968 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:14.947910 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:14.948775 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:14.950555 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:14.951003 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:14.951052 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:14.954748 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:14.954750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:14.954750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:14.954759 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:14.956801 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:14.957445 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:14.957447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:14.957447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:14.957512 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:14.957876 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:14.957878 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:14.957878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:14.958081 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:14.958126 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:14.958614 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 10:10:14.958713 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 10:10:14.959966 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:14.943715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36937 (local address 127.2.30.130:33539)
0808 10:10:14.944788 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:14.944843 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:14.945723 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:14.947129 (+ 1406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:14.947135 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:14.947439 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:14.947758 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:14.947766 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:14.948929 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:14.948930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:14.950442 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:14.950446 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:14.951161 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:14.951162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:14.951392 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:14.952302 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:14.952314 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:14.954606 (+ 2292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:14.956931 (+ 2325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:14.956940 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:14.956957 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:14.957325 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:14.957590 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:14.957591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:14.957592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:14.957747 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:14.958189 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:14.958196 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:14.958431 (+ 235us) client_negotiation.cc:769] Sending connection context
0808 10:10:14.959330 (+ 899us) client_negotiation.cc:240] Negotiation successful
0808 10:10:14.959571 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1015,"thread_start_us":668,"threads_started":1}
I20260808 10:10:14.960939 15266 heartbeater.cc:347] Connected to a master server at 127.2.30.190:36937
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
I20260808 10:10:14.961061 15266 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:14.961284 15266 heartbeater.cc:511] Master 127.2.30.190:36937 requested a full tablet report, sending...
I20260808 10:10:14.961781 14901 ts_manager.cc:194] Registered new tserver with Master: fb7601ed0384490684d4be0e4b5c6535 (127.2.30.130:33417)
I20260808 10:10:14.962601 14901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:33539
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:10:14.987537 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.30.148:46191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36937
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:15.147187 15271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:15.147522 15271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:15.147562 15271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:15.149330 15271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:15.149389 15271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:15.149431 15271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:10:15.151727 15271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.30.148:46191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:15.152014 15271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:15.152335 15271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:15.160629 15278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:15.160857 15280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:15.163242 15277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:15.163801 15271 server_base.cc:1034] running on GCE node
I20260808 10:10:15.164109 15271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:15.164458 15271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:15.165642 15271 hybrid_clock.cc:648] HybridClock initialized: now 1786183815165581 us; error 76 us; skew 500 ppm
Aug 08 10:10:15 dist-test-slave-w206 krb5kdc[14401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183815, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:15.169711 15271 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:10:15.170286 15271 webserver.cc:460] Webserver started at http://127.2.30.131:46345/ using document root <none> and password file <none>
I20260808 10:10:15.170703 15271 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:15.170760 15271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:15.170897 15271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 10:10:15 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183815, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
I20260808 10:10:15.197788 15271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "05527fec89c245e1a72baefc6cb5792b"
format_stamp: "Formatted at 2026-08-08 10:10:15 on dist-test-slave-w206"
server_key: "033cd6ec0bc7f81cbff3a789e446b09e"
server_key_iv: "12aa7cd90be4e30e35a757a4f7e5a850"
server_key_version: "kuduclusterkey@0"
I20260808 10:10:15.198658 15271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "05527fec89c245e1a72baefc6cb5792b"
format_stamp: "Formatted at 2026-08-08 10:10:15 on dist-test-slave-w206"
server_key: "033cd6ec0bc7f81cbff3a789e446b09e"
server_key_iv: "12aa7cd90be4e30e35a757a4f7e5a850"
server_key_version: "kuduclusterkey@0"
I20260808 10:10:15.208504 15271 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.002s
I20260808 10:10:15.224406 15287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:15.224606 15271 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:15.224701 15271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "05527fec89c245e1a72baefc6cb5792b"
format_stamp: "Formatted at 2026-08-08 10:10:15 on dist-test-slave-w206"
server_key: "033cd6ec0bc7f81cbff3a789e446b09e"
server_key_iv: "12aa7cd90be4e30e35a757a4f7e5a850"
server_key_version: "kuduclusterkey@0"
I20260808 10:10:15.224803 15271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:15.263154 15271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:15.266188 15271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:15.266355 15271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:15.266785 15271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:15.267378 15271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:10:15.267483 15271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:15.267524 15271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:10:15.267544 15271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:15.280105 15271 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:45325
I20260808 10:10:15.280624 15271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:10:15.287411 15405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:45325 every 8 connection(s)
I20260808 10:10:15.289448 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 15271
I20260808 10:10:15.289559 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:10:15.289731 2170 external_mini_cluster.cc:1428] Setting key 2916fcc621edd23695d98da3ce6c9ab4
Aug 08 10:10:15 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183815, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:15.307926 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:15.287082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:47553 (local address 127.2.30.190:36937)
0808 10:10:15.287140 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:15.287143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:15.288258 (+ 1115us) server_negotiation.cc:407] Connection header received
0808 10:10:15.289161 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:15.289163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:15.289181 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:15.289219 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:15.290281 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:15.291196 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:15.293412 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:15.293745 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:15.293796 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:15.303736 (+ 9940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:15.303738 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:15.303739 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:15.303749 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:15.305828 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:15.306524 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:15.306526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:15.306526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:15.306596 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:15.306955 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:15.306956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:15.306956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:15.307157 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:15.307210 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:15.307686 (+ 476us) server_negotiation.cc:299] Negotiation successful
0808 10:10:15.307806 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:10:15.308835 15408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:15.286700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36937 (local address 127.2.30.131:47553)
0808 10:10:15.288078 (+ 1378us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:15.288131 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:15.289005 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:15.289448 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:15.289453 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:15.289757 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:15.290109 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:15.290116 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:15.291797 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:15.291799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:15.293287 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:15.293290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:15.300181 (+ 6891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:15.300183 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:15.300448 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:15.301334 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:15.301346 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:15.303585 (+ 2239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:15.305982 (+ 2397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:15.305993 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:15.305997 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:15.306406 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:15.306682 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:15.306683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:15.306683 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:15.306840 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:15.307272 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:15.307280 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:15.307496 (+ 216us) client_negotiation.cc:769] Sending connection context
0808 10:10:15.308360 (+ 864us) client_negotiation.cc:240] Negotiation successful
0808 10:10:15.308563 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1325,"thread_start_us":109,"threads_started":1}
I20260808 10:10:15.309621 15406 heartbeater.cc:347] Connected to a master server at 127.2.30.190:36937
I20260808 10:10:15.309731 15406 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:15.309970 15406 heartbeater.cc:511] Master 127.2.30.190:36937 requested a full tablet report, sending...
I20260808 10:10:15.310478 14901 ts_manager.cc:194] Registered new tserver with Master: 05527fec89c245e1a72baefc6cb5792b (127.2.30.131:45325)
I20260808 10:10:15.311075 14901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:47553
I20260808 10:10:15.314183 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:10:15.314922 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 14988
I20260808 10:10:15.336187 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 15131
I20260808 10:10:15.346833 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 15271
I20260808 10:10:15.356665 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 14868
I20260808 10:10:15.365262 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:36937
--webserver_interface=127.2.30.190
--webserver_port=43037
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:36937
--ranger_config_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.30.148:46191/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.30.148:46191/kms with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:15.450623 15409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:15.450889 15409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:15.450929 15409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:15.452162 15409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:10:15.452234 15409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:15.452258 15409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:15.452280 15409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:10:15.452324 15409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:10:15.454113 15409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.30.148:46191/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:36937
--ranger_config_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:36937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=43037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:15.454409 15409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:15.454653 15409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:15.460004 15414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:15.460007 15409 server_base.cc:1034] running on GCE node
W20260808 10:10:15.460028 15417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:15.460024 15415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:15.460573 15409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:15.460906 15409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:15.462064 15409 hybrid_clock.cc:648] HybridClock initialized: now 1786183815462041 us; error 47 us; skew 500 ppm
Aug 08 10:10:15 dist-test-slave-w206 krb5kdc[14401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183815, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:15.465233 15409 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:10:15.465736 15409 webserver.cc:460] Webserver started at http://127.2.30.190:43037/ using document root <none> and password file <none>
I20260808 10:10:15.466002 15409 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:15.466058 15409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:10:15 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183815, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
I20260808 10:10:15.512657 15409 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:15.514252 15426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:15.514494 15409 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:15.514617 15409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "18f937798818495cafadca0ccee59eb5"
format_stamp: "Formatted at 2026-08-08 10:10:13 on dist-test-slave-w206"
server_key: "65c2d823972c848fd98d57059f42b3ce"
server_key_iv: "920a533f979446651359a52c843e2ac5"
server_key_version: "kuduclusterkey@0"
I20260808 10:10:15.515043 15409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:15.528368 15409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:15.530032 15409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:15.530211 15409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:15.535032 15409 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:36937
I20260808 10:10:15.535166 15480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:36937 every 8 connection(s)
I20260808 10:10:15.535513 15409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:10:15.537070 15481 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:10:15.537846 15481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5: Bootstrap starting.
I20260808 10:10:15.541934 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 15409
I20260808 10:10:15.542330 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.30.148:46191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:38567
--local_ip_for_outbound_sockets=127.2.30.129
--tserver_master_addrs=127.2.30.190:36937
--webserver_port=41589
--webserver_interface=127.2.30.129
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.30.148:46191/kms with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:10:15.547700 15481 log.cc:824] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:15.549774 15481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:10:15.550149 15481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5: Bootstrap complete.
I20260808 10:10:15.553056 15481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f937798818495cafadca0ccee59eb5" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36937 } }
I20260808 10:10:15.553381 15481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18f937798818495cafadca0ccee59eb5, State: Initialized, Role: FOLLOWER
I20260808 10:10:15.553628 15481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f937798818495cafadca0ccee59eb5" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36937 } }
I20260808 10:10:15.553753 15481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:10:15.553820 15481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:10:15.553880 15481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:10:15.555182 15481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f937798818495cafadca0ccee59eb5" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36937 } }
I20260808 10:10:15.555323 15481 leader_election.cc:302] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18f937798818495cafadca0ccee59eb5; no voters:
I20260808 10:10:15.555546 15481 leader_election.cc:288] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:10:15.555626 15485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:10:15.555825 15485 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [term 2 LEADER]: Becoming Leader. State: Replica: 18f937798818495cafadca0ccee59eb5, State: Running, Role: LEADER
I20260808 10:10:15.555896 15481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:10:15.556001 15485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f937798818495cafadca0ccee59eb5" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36937 } }
I20260808 10:10:15.556288 15485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18f937798818495cafadca0ccee59eb5. Latest consensus state: current_term: 2 leader_uuid: "18f937798818495cafadca0ccee59eb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f937798818495cafadca0ccee59eb5" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36937 } } }
I20260808 10:10:15.556370 15485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [sys.catalog]: This master's current role is: LEADER
I20260808 10:10:15.556290 15487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "18f937798818495cafadca0ccee59eb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18f937798818495cafadca0ccee59eb5" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36937 } } }
I20260808 10:10:15.556422 15487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5 [sys.catalog]: This master's current role is: LEADER
I20260808 10:10:15.557082 15491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:10:15.557637 15491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:10:15.557791 15491 catalog_manager.cc:1259] Loaded cluster ID: 9356baacc7214736b97dbf55b026f9bf
I20260808 10:10:15.557832 15491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:10:15.558204 15481 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 10:10:15.561848 15491 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:10:15.562414 15491 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5: Loaded TSK: 0
I20260808 10:10:15.562727 15491 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:10:15.642201 15483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:15.642446 15483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:15.642489 15483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:15.643778 15483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:15.643850 15483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:15.643895 15483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:10:15.646147 15483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.30.148:46191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:38567
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=41589
--tserver_master_addrs=127.2.30.190:36937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:15.646447 15483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:15.646739 15483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:15.654055 15520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:15.654066 15483 server_base.cc:1034] running on GCE node
W20260808 10:10:15.654021 15519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:15.654032 15522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:15.654537 15483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:15.654862 15483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:15.658145 15483 hybrid_clock.cc:648] HybridClock initialized: now 1786183815658124 us; error 55 us; skew 500 ppm
Aug 08 10:10:15 dist-test-slave-w206 krb5kdc[14401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183815, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:15.661212 15483 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:10:15.661816 15483 webserver.cc:460] Webserver started at http://127.2.30.129:41589/ using document root <none> and password file <none>
I20260808 10:10:15.662070 15483 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:15.662153 15483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:10:15 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183815, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
I20260808 10:10:15.707461 15483 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.001s sys 0.000s
I20260808 10:10:15.708722 15531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:15.708950 15483 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:15.709079 15483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "b11e6b67d1854791b8a394a096e2a3f3"
format_stamp: "Formatted at 2026-08-08 10:10:14 on dist-test-slave-w206"
server_key: "886b056c4be4ce01b446c05be1999276"
server_key_iv: "3ea2e562c5a5a040f8f2cd0e5914b813"
server_key_version: "kuduclusterkey@0"
I20260808 10:10:15.709563 15483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:15.722924 15483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:15.725111 15483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:15.725303 15483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:15.725584 15483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:15.726138 15483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:10:15.726271 15483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:15.726322 15483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:10:15.726354 15483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:15.734411 15483 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:38567
I20260808 10:10:15.734918 15483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:10:15.736035 15647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:38567 every 8 connection(s)
I20260808 10:10:15.739796 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 15483
I20260808 10:10:15.740170 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.30.148:46191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:33417
--local_ip_for_outbound_sockets=127.2.30.130
--tserver_master_addrs=127.2.30.190:36937
--webserver_port=44325
--webserver_interface=127.2.30.130
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.30.148:46191/kms with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:10:15 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183815, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:15.752475 15651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:15.737030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:55493 (local address 127.2.30.190:36937)
0808 10:10:15.739573 (+ 2543us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:15.739585 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:15.739635 (+ 50us) server_negotiation.cc:407] Connection header received
0808 10:10:15.739814 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:15.739822 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:15.739924 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:15.740913 (+ 989us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:15.741020 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:15.742470 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:15.744043 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:15.744684 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:15.745326 (+ 642us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:15.748060 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:15.748065 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:15.748066 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:15.748079 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:15.750301 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:15.750849 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:15.750853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:15.750853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:15.750967 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:15.751347 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:15.751348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:15.751348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:15.751666 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:15.751782 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:15.752097 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 10:10:15.752216 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2452,"thread_start_us":95,"threads_started":1}
I20260808 10:10:15.752466 15650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:15.736937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36937 (local address 127.2.30.129:55493)
0808 10:10:15.737365 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:15.737415 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:15.738067 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:15.740179 (+ 2112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:15.740185 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:15.740460 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:15.740759 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:15.740766 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:15.742610 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:15.742612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:15.743916 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:15.743920 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:15.744840 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:15.744842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:15.744994 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:15.745725 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:15.745734 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:15.747868 (+ 2134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:15.750411 (+ 2543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:15.750419 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:15.750421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:15.750741 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:15.751080 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:15.751081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:15.751081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:15.751261 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:15.751760 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:15.751767 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:15.751920 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 10:10:15.752041 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 10:10:15.752216 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":100,"threads_started":1}
I20260808 10:10:15.754122 15648 heartbeater.cc:347] Connected to a master server at 127.2.30.190:36937
I20260808 10:10:15.754276 15648 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:15.754475 15648 heartbeater.cc:511] Master 127.2.30.190:36937 requested a full tablet report, sending...
W20260808 10:10:15.755426 15648 heartbeater.cc:644] Failed to heartbeat to 127.2.30.190:36937 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 10:10:15.755505 15648 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:15.755578 15648 heartbeater.cc:511] Master 127.2.30.190:36937 requested a full tablet report, sending...
I20260808 10:10:15.756013 15648 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:15.756098 15648 heartbeater.cc:511] Master 127.2.30.190:36937 requested a full tablet report, sending...
W20260808 10:10:15.756448 15648 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:10:15.876154 15652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:15.876490 15652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:15.876544 15652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:15.877983 15652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:15.878054 15652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:15.878108 15652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:10:15.880296 15652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.30.148:46191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:33417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=44325
--tserver_master_addrs=127.2.30.190:36937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:15.880620 15652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:15.880913 15652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:15.891477 15660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:15.891491 15658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:15.892186 15657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:15.892355 15652 server_base.cc:1034] running on GCE node
I20260808 10:10:15.892591 15652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:15.892918 15652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:15.894099 15652 hybrid_clock.cc:648] HybridClock initialized: now 1786183815894079 us; error 37 us; skew 500 ppm
Aug 08 10:10:15 dist-test-slave-w206 krb5kdc[14401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183815, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:15.898669 15652 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:10:15.899510 15652 webserver.cc:460] Webserver started at http://127.2.30.130:44325/ using document root <none> and password file <none>
I20260808 10:10:15.899931 15652 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:15.900053 15652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:10:15 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183815, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
I20260808 10:10:15.921793 15652 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:10:15.928834 15668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:15.929139 15652 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260808 10:10:15.929301 15652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "fb7601ed0384490684d4be0e4b5c6535"
format_stamp: "Formatted at 2026-08-08 10:10:14 on dist-test-slave-w206"
server_key: "e69462754b0d0c6bd60631883d1416e7"
server_key_iv: "fd7a9118994c84c609cd3ffb35d83ab1"
server_key_version: "kuduclusterkey@0"
I20260808 10:10:15.929884 15652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:15.968070 15652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:15.971047 15652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:15.971269 15652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:15.971602 15652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:15.972065 15652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:10:15.972126 15652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:15.972163 15652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:10:15.972203 15652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:15.981029 15652 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:33417
I20260808 10:10:15.981680 15652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:10:15.982412 15787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:33417 every 8 connection(s)
I20260808 10:10:15.989523 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 15652
I20260808 10:10:15.990012 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.30.148:46191/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:45325
--local_ip_for_outbound_sockets=127.2.30.131
--tserver_master_addrs=127.2.30.190:36937
--webserver_port=46345
--webserver_interface=127.2.30.131
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.30.148:46191/kms with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:10:15 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183815, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:15.995673 15651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:15.983524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:47969 (local address 127.2.30.190:36937)
0808 10:10:15.983584 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:15.983586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:15.984044 (+ 458us) server_negotiation.cc:407] Connection header received
0808 10:10:15.984883 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:15.984885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:15.984902 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:15.984939 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:15.985907 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:15.986466 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:15.988364 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:15.988566 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:15.988632 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:15.991654 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:15.991656 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:15.991656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:15.991666 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:15.993602 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:15.994237 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:15.994239 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:15.994239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:15.994313 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:15.994681 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:15.994683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:15.994683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:15.994868 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:15.994947 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:15.995410 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 10:10:15.995526 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 10:10:15.996239 15790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:15.983129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36937 (local address 127.2.30.130:47969)
0808 10:10:15.983889 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:15.983940 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:15.984729 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:15.985196 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:15.985200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:15.985465 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:15.985782 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:15.985789 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:15.986603 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:15.986604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:15.988254 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:15.988258 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:15.988694 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:15.988695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:15.988854 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:15.989571 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:15.989582 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:15.991506 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:15.993721 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:15.993731 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:15.993734 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:15.994103 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:15.994403 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:15.994405 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:15.994405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:15.994576 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:15.994961 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:15.994970 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:15.995214 (+ 244us) client_negotiation.cc:769] Sending connection context
0808 10:10:15.995345 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 10:10:15.995539 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":110,"threads_started":1}
I20260808 10:10:15.997071 15788 heartbeater.cc:347] Connected to a master server at 127.2.30.190:36937
I20260808 10:10:15.997198 15788 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:15.997442 15788 heartbeater.cc:511] Master 127.2.30.190:36937 requested a full tablet report, sending...
W20260808 10:10:15.998345 15788 heartbeater.cc:644] Failed to heartbeat to 127.2.30.190:36937 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 10:10:15.998435 15788 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:15.998519 15788 heartbeater.cc:511] Master 127.2.30.190:36937 requested a full tablet report, sending...
I20260808 10:10:15.998934 15788 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:15.999058 15788 heartbeater.cc:511] Master 127.2.30.190:36937 requested a full tablet report, sending...
W20260808 10:10:15.999593 15788 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:10:16.103322 15791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:16.103658 15791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:16.103710 15791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:16.105038 15791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:16.105124 15791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:16.105186 15791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:10:16.107110 15791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.30.148:46191/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:45325
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=46345
--tserver_master_addrs=127.2.30.190:36937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:16.107522 15791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:16.107831 15791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:16.113830 15799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:16.113830 15796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:16.114399 15797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:16.115059 15791 server_base.cc:1034] running on GCE node
I20260808 10:10:16.115374 15791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:16.115765 15791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:16.120347 15791 hybrid_clock.cc:648] HybridClock initialized: now 1786183816120321 us; error 36 us; skew 500 ppm
Aug 08 10:10:16 dist-test-slave-w206 krb5kdc[14401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183816, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:16.124526 15791 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:10:16.125262 15791 webserver.cc:460] Webserver started at http://127.2.30.131:46345/ using document root <none> and password file <none>
I20260808 10:10:16.125633 15791 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:16.125797 15791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 10:10:16 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183816, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
I20260808 10:10:16.152675 15791 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:16.154060 15808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:16.154321 15791 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:16.154419 15791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "05527fec89c245e1a72baefc6cb5792b"
format_stamp: "Formatted at 2026-08-08 10:10:15 on dist-test-slave-w206"
server_key: "033cd6ec0bc7f81cbff3a789e446b09e"
server_key_iv: "12aa7cd90be4e30e35a757a4f7e5a850"
server_key_version: "kuduclusterkey@0"
I20260808 10:10:16.154937 15791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:16.184628 15791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:16.187573 15791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:16.187765 15791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:16.188031 15791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:16.188480 15791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:10:16.188549 15791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:16.188598 15791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:10:16.188628 15791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:16.198498 15791 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:45325
I20260808 10:10:16.198599 15927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:45325 every 8 connection(s)
I20260808 10:10:16.199059 15791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:10:16.202255 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 15791
Aug 08 10:10:16 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183816, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:16.211623 15651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:16.199977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:40199 (local address 127.2.30.190:36937)
0808 10:10:16.200033 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:16.200036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:16.200387 (+ 351us) server_negotiation.cc:407] Connection header received
0808 10:10:16.201264 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:16.201265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:16.201280 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:16.201348 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:16.202207 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:16.203011 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:16.204783 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:16.205007 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:16.205100 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:16.207950 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:16.207951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:16.207952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:16.207961 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:16.209751 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:16.210397 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:16.210399 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:16.210399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:16.210468 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:16.210763 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:16.210764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:16.210765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:16.210911 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:16.211003 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:16.211419 (+ 416us) server_negotiation.cc:299] Negotiation successful
0808 10:10:16.211524 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 10:10:16.212080 15930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:16.199983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36937 (local address 127.2.30.131:40199)
0808 10:10:16.200207 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:16.200262 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:16.200991 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:16.201497 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:16.201502 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:16.201765 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:16.202077 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:16.202084 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:16.203292 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:16.203294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:16.204681 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:16.204685 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:16.205141 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:16.205143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:16.205303 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:16.205957 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:16.205968 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:16.207831 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:16.210021 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:16.210028 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:16.210031 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:16.210306 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:16.210541 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:16.210541 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:16.210542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:16.210688 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:16.211032 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:16.211040 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:16.211238 (+ 198us) client_negotiation.cc:769] Sending connection context
0808 10:10:16.211368 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 10:10:16.211541 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":109,"threads_started":1}
I20260808 10:10:16.212569 15928 heartbeater.cc:347] Connected to a master server at 127.2.30.190:36937
I20260808 10:10:16.212706 15928 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:16.212911 15928 heartbeater.cc:511] Master 127.2.30.190:36937 requested a full tablet report, sending...
W20260808 10:10:16.213699 15928 heartbeater.cc:644] Failed to heartbeat to 127.2.30.190:36937 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 10:10:16.213789 15928 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:16.213868 15928 heartbeater.cc:511] Master 127.2.30.190:36937 requested a full tablet report, sending...
I20260808 10:10:16.214164 15928 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:16.214241 15928 heartbeater.cc:511] Master 127.2.30.190:36937 requested a full tablet report, sending...
W20260808 10:10:16.214560 15928 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:10:16.215247 15932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:16.203054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37882 (local address 127.2.30.190:36937)
0808 10:10:16.203223 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:16.203225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:16.203233 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:10:16.203254 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:16.203255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:16.203267 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:16.203364 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:16.204255 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:16.205081 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:16.206817 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:16.207165 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:16.207251 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:16.208910 (+ 1659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:16.208912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:16.208912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:16.208920 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:16.210777 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:16.211860 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:16.211861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:16.211862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:16.211930 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:16.213939 (+ 2009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:16.213941 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:16.213941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:16.214103 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:16.214302 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:16.215090 (+ 788us) server_negotiation.cc:299] Negotiation successful
0808 10:10:16.215171 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":87,"threads_started":1}
I20260808 10:10:16.756649 15648 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:16.756834 15648 heartbeater.cc:511] Master 127.2.30.190:36937 requested a full tablet report, sending...
I20260808 10:10:16.757328 15443 ts_manager.cc:194] Registered new tserver with Master: b11e6b67d1854791b8a394a096e2a3f3 (127.2.30.129:38567)
I20260808 10:10:16.758257 15443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:55493
Aug 08 10:10:16 dist-test-slave-w206 krb5kdc[14401](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786183816, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:16.999814 15788 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:17.000008 15788 heartbeater.cc:511] Master 127.2.30.190:36937 requested a full tablet report, sending...
I20260808 10:10:17.000476 15443 ts_manager.cc:194] Registered new tserver with Master: fb7601ed0384490684d4be0e4b5c6535 (127.2.30.130:33417)
I20260808 10:10:17.000982 15443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:47969
Aug 08 10:10:17 dist-test-slave-w206 krb5kdc[14401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183816, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for HTTP/127.2.30.148@KRBTEST.COM
I20260808 10:10:17.214754 15928 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:17.214941 15928 heartbeater.cc:511] Master 127.2.30.190:36937 requested a full tablet report, sending...
I20260808 10:10:17.215301 15443 ts_manager.cc:194] Registered new tserver with Master: 05527fec89c245e1a72baefc6cb5792b (127.2.30.131:45325)
I20260808 10:10:17.215728 15443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:40199
I20260808 10:10:17.222216 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:10:17.236246 15949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:17.227420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37898 (local address 127.2.30.190:36937)
0808 10:10:17.227810 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:17.227812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:17.227952 (+ 140us) server_negotiation.cc:407] Connection header received
0808 10:10:17.228070 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:17.228071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:17.228087 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:17.228137 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:17.228936 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.229729 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.230787 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.231130 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.231178 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:17.232814 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:17.232816 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:17.232816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:17.232825 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:17.234754 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:17.235250 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:17.235252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:17.235253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:17.235323 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:17.235545 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:17.235546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:17.235546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:17.235691 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:17.235771 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:17.236028 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 10:10:17.236126 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":92,"threads_started":1}
I20260808 10:10:17.239272 15443 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:10:17.242174 15443 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:10:17.266783 15956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:17.251126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:45325 (local address 127.0.0.1:53046)
0808 10:10:17.252655 (+ 1529us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:17.252664 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:17.252708 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:17.253030 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:17.253034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:17.253036 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:17.253113 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:17.253115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.258724 (+ 5609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.258726 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:17.259990 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:17.259994 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.266398 (+ 6404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.266398 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:17.266511 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:17.266532 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:10:17.266608 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:10:17.266673 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1513,"thread_start_us":73,"threads_started":1}
I20260808 10:10:17.267431 15962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:17.251344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53046 (local address 127.2.30.131:45325)
0808 10:10:17.251961 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:17.251967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:17.252767 (+ 800us) server_negotiation.cc:407] Connection header received
0808 10:10:17.252820 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:17.252822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:17.252836 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:17.252916 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:17.257171 (+ 4255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.258274 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.260111 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.260996 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.261080 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:17.261151 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:17.267278 (+ 6127us) server_negotiation.cc:299] Negotiation successful
0808 10:10:17.267344 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":75,"threads_started":1}
I20260808 10:10:17.268260 15959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:17.251102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39294 (local address 127.2.30.129:38567)
0808 10:10:17.252040 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:17.252044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:17.252218 (+ 174us) server_negotiation.cc:407] Connection header received
0808 10:10:17.252258 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:17.252260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:17.252276 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:17.252346 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:17.253679 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.254931 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.263165 (+ 8234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.264030 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.264106 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:17.264170 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:17.268116 (+ 3946us) server_negotiation.cc:299] Negotiation successful
0808 10:10:17.268180 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":904,"thread_start_us":61,"threads_started":1}
I20260808 10:10:17.268024 15961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:17.250920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:38567 (local address 127.0.0.1:39294)
0808 10:10:17.252101 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:17.252112 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:17.252166 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:17.253547 (+ 1381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:17.253547 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:17.253549 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:17.253591 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:17.253593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.255865 (+ 2272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.255866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:17.263055 (+ 7189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:17.263060 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.267721 (+ 4661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.267722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:17.267826 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:17.267831 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:10:17.267893 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 10:10:17.267943 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1165,"thread_start_us":98,"threads_started":1}
I20260808 10:10:17.269719 15958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:17.250654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:33417 (local address 127.0.0.1:58878)
0808 10:10:17.251655 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:17.251702 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:17.251779 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:17.255146 (+ 3367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:17.255147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:17.255150 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:17.255211 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:17.255213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.264247 (+ 9034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.264249 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:17.265360 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:17.265364 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.269497 (+ 4133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.269499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:17.269571 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:17.269576 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:10:17.269605 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 10:10:17.269646 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"thread_start_us":85,"threads_started":1}
I20260808 10:10:17.269918 15860 tablet_service.cc:1467] Processing CreateTablet for tablet e9f067bdd9584fe7aa01cdc0b8e2de53 (DEFAULT_TABLE table=test-table [id=8985aead749a4e6ea36e286633a35e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:10:17.270027 15566 tablet_service.cc:1467] Processing CreateTablet for tablet e9f067bdd9584fe7aa01cdc0b8e2de53 (DEFAULT_TABLE table=test-table [id=8985aead749a4e6ea36e286633a35e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:10:17.270409 15860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9f067bdd9584fe7aa01cdc0b8e2de53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:17.270465 15566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9f067bdd9584fe7aa01cdc0b8e2de53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:17.270584 15957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:17.250809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58878 (local address 127.2.30.130:33417)
0808 10:10:17.252489 (+ 1680us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:17.252493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:17.252503 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:10:17.252535 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:17.252537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:17.252552 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:17.255307 (+ 2755us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:17.255402 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.262609 (+ 7207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.265483 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.268921 (+ 3438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.270077 (+ 1156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:17.270288 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:17.270323 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 10:10:17.270381 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1644,"thread_start_us":64,"threads_started":1}
I20260808 10:10:17.274015 15702 tablet_service.cc:1467] Processing CreateTablet for tablet e9f067bdd9584fe7aa01cdc0b8e2de53 (DEFAULT_TABLE table=test-table [id=8985aead749a4e6ea36e286633a35e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:10:17.274411 15702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9f067bdd9584fe7aa01cdc0b8e2de53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:17.274855 15963 tablet_bootstrap.cc:492] T e9f067bdd9584fe7aa01cdc0b8e2de53 P b11e6b67d1854791b8a394a096e2a3f3: Bootstrap starting.
I20260808 10:10:17.275573 15963 tablet_bootstrap.cc:654] T e9f067bdd9584fe7aa01cdc0b8e2de53 P b11e6b67d1854791b8a394a096e2a3f3: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:17.275904 15963 log.cc:824] T e9f067bdd9584fe7aa01cdc0b8e2de53 P b11e6b67d1854791b8a394a096e2a3f3: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:17.280750 15965 tablet_bootstrap.cc:492] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535: Bootstrap starting.
I20260808 10:10:17.281534 15965 tablet_bootstrap.cc:654] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:17.281895 15965 log.cc:824] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:17.282862 15963 tablet_bootstrap.cc:492] T e9f067bdd9584fe7aa01cdc0b8e2de53 P b11e6b67d1854791b8a394a096e2a3f3: No bootstrap required, opened a new log
I20260808 10:10:17.282943 15963 ts_tablet_manager.cc:1397] T e9f067bdd9584fe7aa01cdc0b8e2de53 P b11e6b67d1854791b8a394a096e2a3f3: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260808 10:10:17.287024 15963 raft_consensus.cc:348] T e9f067bdd9584fe7aa01cdc0b8e2de53 P b11e6b67d1854791b8a394a096e2a3f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11e6b67d1854791b8a394a096e2a3f3" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 38567 } } peers { permanent_uuid: "fb7601ed0384490684d4be0e4b5c6535" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 33417 } } peers { permanent_uuid: "05527fec89c245e1a72baefc6cb5792b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45325 } }
I20260808 10:10:17.287245 15963 raft_consensus.cc:374] T e9f067bdd9584fe7aa01cdc0b8e2de53 P b11e6b67d1854791b8a394a096e2a3f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:17.287292 15963 raft_consensus.cc:729] T e9f067bdd9584fe7aa01cdc0b8e2de53 P b11e6b67d1854791b8a394a096e2a3f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b11e6b67d1854791b8a394a096e2a3f3, State: Initialized, Role: FOLLOWER
I20260808 10:10:17.287319 15966 tablet_bootstrap.cc:492] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b: Bootstrap starting.
I20260808 10:10:17.287436 15963 consensus_queue.cc:260] T e9f067bdd9584fe7aa01cdc0b8e2de53 P b11e6b67d1854791b8a394a096e2a3f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11e6b67d1854791b8a394a096e2a3f3" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 38567 } } peers { permanent_uuid: "fb7601ed0384490684d4be0e4b5c6535" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 33417 } } peers { permanent_uuid: "05527fec89c245e1a72baefc6cb5792b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45325 } }
I20260808 10:10:17.287801 15963 ts_tablet_manager.cc:1428] T e9f067bdd9584fe7aa01cdc0b8e2de53 P b11e6b67d1854791b8a394a096e2a3f3: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260808 10:10:17.288009 15648 heartbeater.cc:503] Master 127.2.30.190:36937 was elected leader, sending a full tablet report...
I20260808 10:10:17.288427 15966 tablet_bootstrap.cc:654] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:17.288978 15965 tablet_bootstrap.cc:492] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535: No bootstrap required, opened a new log
I20260808 10:10:17.289065 15965 ts_tablet_manager.cc:1397] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260808 10:10:17.289206 15966 log.cc:824] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:17.293226 15965 raft_consensus.cc:348] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11e6b67d1854791b8a394a096e2a3f3" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 38567 } } peers { permanent_uuid: "fb7601ed0384490684d4be0e4b5c6535" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 33417 } } peers { permanent_uuid: "05527fec89c245e1a72baefc6cb5792b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45325 } }
I20260808 10:10:17.293455 15965 raft_consensus.cc:374] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:17.293509 15965 raft_consensus.cc:729] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb7601ed0384490684d4be0e4b5c6535, State: Initialized, Role: FOLLOWER
I20260808 10:10:17.293663 15965 consensus_queue.cc:260] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11e6b67d1854791b8a394a096e2a3f3" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 38567 } } peers { permanent_uuid: "fb7601ed0384490684d4be0e4b5c6535" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 33417 } } peers { permanent_uuid: "05527fec89c245e1a72baefc6cb5792b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45325 } }
I20260808 10:10:17.294077 15965 ts_tablet_manager.cc:1428] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 10:10:17.294214 15788 heartbeater.cc:503] Master 127.2.30.190:36937 was elected leader, sending a full tablet report...
I20260808 10:10:17.295555 15966 tablet_bootstrap.cc:492] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b: No bootstrap required, opened a new log
I20260808 10:10:17.295643 15966 ts_tablet_manager.cc:1397] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260808 10:10:17.299916 15966 raft_consensus.cc:348] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11e6b67d1854791b8a394a096e2a3f3" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 38567 } } peers { permanent_uuid: "fb7601ed0384490684d4be0e4b5c6535" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 33417 } } peers { permanent_uuid: "05527fec89c245e1a72baefc6cb5792b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45325 } }
I20260808 10:10:17.300137 15966 raft_consensus.cc:374] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:17.300187 15966 raft_consensus.cc:729] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05527fec89c245e1a72baefc6cb5792b, State: Initialized, Role: FOLLOWER
I20260808 10:10:17.300357 15966 consensus_queue.cc:260] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11e6b67d1854791b8a394a096e2a3f3" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 38567 } } peers { permanent_uuid: "fb7601ed0384490684d4be0e4b5c6535" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 33417 } } peers { permanent_uuid: "05527fec89c245e1a72baefc6cb5792b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45325 } }
I20260808 10:10:17.300750 15966 ts_tablet_manager.cc:1428] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 10:10:17.300889 15928 heartbeater.cc:503] Master 127.2.30.190:36937 was elected leader, sending a full tablet report...
I20260808 10:10:17.367050 15971 raft_consensus.cc:482] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:10:17.367285 15971 raft_consensus.cc:504] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11e6b67d1854791b8a394a096e2a3f3" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 38567 } } peers { permanent_uuid: "fb7601ed0384490684d4be0e4b5c6535" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 33417 } } peers { permanent_uuid: "05527fec89c245e1a72baefc6cb5792b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45325 } }
I20260808 10:10:17.367834 15971 leader_election.cc:288] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b11e6b67d1854791b8a394a096e2a3f3 (127.2.30.129:38567), fb7601ed0384490684d4be0e4b5c6535 (127.2.30.130:33417)
I20260808 10:10:17.374250 15959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:17.368442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:41263 (local address 127.2.30.129:38567)
0808 10:10:17.368498 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:17.368501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:17.369228 (+ 727us) server_negotiation.cc:407] Connection header received
0808 10:10:17.369441 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:17.369442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:17.369458 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:17.369493 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:17.369940 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.370838 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.372817 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.373511 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.373571 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:17.373582 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:17.374100 (+ 518us) server_negotiation.cc:299] Negotiation successful
0808 10:10:17.374157 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 10:10:17.374761 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:17.368100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:38567 (local address 127.2.30.131:41263)
0808 10:10:17.369137 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:17.369151 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:17.369336 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:17.369572 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:17.369572 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:17.369575 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:17.369819 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:17.369823 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.371010 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.371011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:17.372695 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:17.372698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.373798 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.373800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:17.373935 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:17.373941 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:10:17.374404 (+ 463us) client_negotiation.cc:240] Negotiation successful
0808 10:10:17.374456 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"thread_start_us":145,"threads_started":1}
I20260808 10:10:17.375418 15589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9f067bdd9584fe7aa01cdc0b8e2de53" candidate_uuid: "05527fec89c245e1a72baefc6cb5792b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b11e6b67d1854791b8a394a096e2a3f3" is_pre_election: true
I20260808 10:10:17.375687 15589 raft_consensus.cc:2454] T e9f067bdd9584fe7aa01cdc0b8e2de53 P b11e6b67d1854791b8a394a096e2a3f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05527fec89c245e1a72baefc6cb5792b in term 0.
I20260808 10:10:17.376039 15810 leader_election.cc:302] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05527fec89c245e1a72baefc6cb5792b, b11e6b67d1854791b8a394a096e2a3f3; no voters:
I20260808 10:10:17.376207 15971 raft_consensus.cc:2781] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:10:17.376261 15971 raft_consensus.cc:482] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:10:17.376289 15971 raft_consensus.cc:3037] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:17.378235 15971 raft_consensus.cc:504] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11e6b67d1854791b8a394a096e2a3f3" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 38567 } } peers { permanent_uuid: "fb7601ed0384490684d4be0e4b5c6535" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 33417 } } peers { permanent_uuid: "05527fec89c245e1a72baefc6cb5792b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45325 } }
I20260808 10:10:17.378391 15971 leader_election.cc:288] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [CANDIDATE]: Term 1 election: Requested vote from peers b11e6b67d1854791b8a394a096e2a3f3 (127.2.30.129:38567), fb7601ed0384490684d4be0e4b5c6535 (127.2.30.130:33417)
I20260808 10:10:17.378623 15589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9f067bdd9584fe7aa01cdc0b8e2de53" candidate_uuid: "05527fec89c245e1a72baefc6cb5792b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b11e6b67d1854791b8a394a096e2a3f3"
I20260808 10:10:17.378736 15589 raft_consensus.cc:3037] T e9f067bdd9584fe7aa01cdc0b8e2de53 P b11e6b67d1854791b8a394a096e2a3f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:17.380182 15589 raft_consensus.cc:2454] T e9f067bdd9584fe7aa01cdc0b8e2de53 P b11e6b67d1854791b8a394a096e2a3f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05527fec89c245e1a72baefc6cb5792b in term 1.
I20260808 10:10:17.380401 15810 leader_election.cc:302] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05527fec89c245e1a72baefc6cb5792b, b11e6b67d1854791b8a394a096e2a3f3; no voters:
I20260808 10:10:17.380510 15971 raft_consensus.cc:2781] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:10:17.380697 15971 raft_consensus.cc:686] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [term 1 LEADER]: Becoming Leader. State: Replica: 05527fec89c245e1a72baefc6cb5792b, State: Running, Role: LEADER
I20260808 10:10:17.380818 15971 consensus_queue.cc:237] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11e6b67d1854791b8a394a096e2a3f3" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 38567 } } peers { permanent_uuid: "fb7601ed0384490684d4be0e4b5c6535" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 33417 } } peers { permanent_uuid: "05527fec89c245e1a72baefc6cb5792b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45325 } }
I20260808 10:10:17.381963 15443 catalog_manager.cc:5495] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b reported cstate change: term changed from 0 to 1, leader changed from <none> to 05527fec89c245e1a72baefc6cb5792b (127.2.30.131). New cstate: current_term: 1 leader_uuid: "05527fec89c245e1a72baefc6cb5792b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b11e6b67d1854791b8a394a096e2a3f3" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 38567 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb7601ed0384490684d4be0e4b5c6535" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 33417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05527fec89c245e1a72baefc6cb5792b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 45325 } health_report { overall_health: HEALTHY } } }
I20260808 10:10:17.385097 15957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:17.368971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:48169 (local address 127.2.30.130:33417)
0808 10:10:17.369019 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:17.369021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:17.375238 (+ 6217us) server_negotiation.cc:407] Connection header received
0808 10:10:17.375403 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:17.375404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:17.375421 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:17.375459 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:17.375852 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.376593 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.383673 (+ 7080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.384459 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.384528 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:17.384539 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:17.384950 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 10:10:17.385009 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 10:10:17.385484 15972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:17.368648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:33417 (local address 127.2.30.131:48169)
0808 10:10:17.375107 (+ 6459us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:17.375121 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:17.375285 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:17.375542 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:17.375543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:17.375545 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:17.375733 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:17.375736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.376743 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.376744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:17.378467 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:17.378471 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.384617 (+ 6146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.384618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:17.384826 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:17.384833 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:10:17.385253 (+ 420us) client_negotiation.cc:240] Negotiation successful
0808 10:10:17.385312 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6426,"thread_start_us":163,"threads_started":1}
I20260808 10:10:17.386190 15741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9f067bdd9584fe7aa01cdc0b8e2de53" candidate_uuid: "05527fec89c245e1a72baefc6cb5792b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb7601ed0384490684d4be0e4b5c6535" is_pre_election: true
I20260808 10:10:17.386196 15740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9f067bdd9584fe7aa01cdc0b8e2de53" candidate_uuid: "05527fec89c245e1a72baefc6cb5792b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb7601ed0384490684d4be0e4b5c6535"
I20260808 10:10:17.386374 15741 raft_consensus.cc:2454] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05527fec89c245e1a72baefc6cb5792b in term 0.
I20260808 10:10:17.410035 15962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:17.396600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53062 (local address 127.2.30.131:45325)
0808 10:10:17.396660 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:17.396662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:17.396712 (+ 50us) server_negotiation.cc:407] Connection header received
0808 10:10:17.396841 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:17.396842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:17.396873 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:17.396962 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:10:17.397296 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.398135 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.406837 (+ 8702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:17.407211 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:17.409035 (+ 1824us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:17.409395 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:10:17.409547 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:10:17.409625 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:17.409916 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 10:10:17.409949 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:10:17.414325 15740 raft_consensus.cc:3037] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:17.415588 15740 raft_consensus.cc:1261] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535 [term 1 FOLLOWER]: Refusing update from remote peer 05527fec89c245e1a72baefc6cb5792b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:10:17.415972 15974 consensus_queue.cc:1035] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb7601ed0384490684d4be0e4b5c6535" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 33417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:10:17.418465 15589 raft_consensus.cc:1261] T e9f067bdd9584fe7aa01cdc0b8e2de53 P b11e6b67d1854791b8a394a096e2a3f3 [term 1 FOLLOWER]: Refusing update from remote peer 05527fec89c245e1a72baefc6cb5792b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:10:17.418758 15974 consensus_queue.cc:1035] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b11e6b67d1854791b8a394a096e2a3f3" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 38567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:10:17.423671 15980 mvcc.cc:205] Tried to move back new op lower bound from 7316208916125667328 to 7316208915992489984. Current Snapshot: MvccSnapshot[applied={T|T < 7316208916125667328}]
I20260808 10:10:17.424358 15977 mvcc.cc:205] Tried to move back new op lower bound from 7316208916125667328 to 7316208915992489984. Current Snapshot: MvccSnapshot[applied={T|T < 7316208916125667328 or (T in {7316208916125667328})}]
I20260808 10:10:17.424710 15978 mvcc.cc:205] Tried to move back new op lower bound from 7316208916125667328 to 7316208915992489984. Current Snapshot: MvccSnapshot[applied={T|T < 7316208916125667328}]
I20260808 10:10:17.427037 15443 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:10:17.427153 15443 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:10:17.428007 15443 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18f937798818495cafadca0ccee59eb5: Sending DeleteTablet for 3 replicas of tablet e9f067bdd9584fe7aa01cdc0b8e2de53
I20260808 10:10:17.428557 15702 tablet_service.cc:1514] Processing DeleteTablet for tablet e9f067bdd9584fe7aa01cdc0b8e2de53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:10:17 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:58878
I20260808 10:10:17.429173 15860 tablet_service.cc:1514] Processing DeleteTablet for tablet e9f067bdd9584fe7aa01cdc0b8e2de53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:10:17 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:53046
I20260808 10:10:17.429209 15566 tablet_service.cc:1514] Processing DeleteTablet for tablet e9f067bdd9584fe7aa01cdc0b8e2de53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:10:17 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:39294
I20260808 10:10:17.429901 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 15483
I20260808 10:10:17.435702 15987 tablet_replica.cc:330] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b: stopping tablet replica
I20260808 10:10:17.435873 15987 raft_consensus.cc:2229] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:10:17.436030 15987 raft_consensus.cc:2258] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:10:17.436231 15986 tablet_replica.cc:330] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535: stopping tablet replica
I20260808 10:10:17.436363 15986 raft_consensus.cc:2229] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:10:17.436489 15986 raft_consensus.cc:2258] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:10:17.437083 15987 ts_tablet_manager.cc:1905] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:10:17.439517 15986 ts_tablet_manager.cc:1905] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:10:17.447160 15987 ts_tablet_manager.cc:1918] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:10:17.447283 15987 log.cc:1196] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/e9f067bdd9584fe7aa01cdc0b8e2de53
I20260808 10:10:17.447650 15987 ts_tablet_manager.cc:1939] T e9f067bdd9584fe7aa01cdc0b8e2de53 P 05527fec89c245e1a72baefc6cb5792b: Deleting consensus metadata
I20260808 10:10:17.447911 15986 ts_tablet_manager.cc:1918] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:10:17.448009 15986 log.cc:1196] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/e9f067bdd9584fe7aa01cdc0b8e2de53
I20260808 10:10:17.448294 15986 ts_tablet_manager.cc:1939] T e9f067bdd9584fe7aa01cdc0b8e2de53 P fb7601ed0384490684d4be0e4b5c6535: Deleting consensus metadata
I20260808 10:10:17.448904 15430 catalog_manager.cc:4841] TS fb7601ed0384490684d4be0e4b5c6535 (127.2.30.130:33417): tablet e9f067bdd9584fe7aa01cdc0b8e2de53 (table test-table [id=8985aead749a4e6ea36e286633a35e0c]) successfully deleted
I20260808 10:10:17.449370 15428 catalog_manager.cc:4841] TS 05527fec89c245e1a72baefc6cb5792b (127.2.30.131:45325): tablet e9f067bdd9584fe7aa01cdc0b8e2de53 (table test-table [id=8985aead749a4e6ea36e286633a35e0c]) successfully deleted
W20260808 10:10:17.449882 15428 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:38567 (error 108)
I20260808 10:10:17.450641 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 15652
I20260808 10:10:17.471436 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 15791
I20260808 10:10:17.473690 15958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:17.473507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:38567 (local address 127.0.0.1:39296)
0808 10:10:17.473582 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:17.473637 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:38567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 10:10:17.473977 15428 catalog_manager.cc:4584] TS b11e6b67d1854791b8a394a096e2a3f3 (127.2.30.129:38567): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e9f067bdd9584fe7aa01cdc0b8e2de53: Network error: Client connection negotiation failed: client connection to 127.2.30.129:38567: connect: Connection refused (error 111)
I20260808 10:10:17.479624 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 15409
I20260808 10:10:17.488273 2170 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 10:10:17.833101 2170 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 10:10:17.833194 2170 mini_ranger.cc:66] Stopping Ranger...
I20260808 10:10:18.188454 2170 mini_ranger.cc:68] Stopped Ranger
2026-08-08 10:10:18.188 UTC [14431] LOG: received smart shutdown request
2026-08-08 10:10:18.193 UTC [14431] LOG: background worker "logical replication launcher" (PID 14443) exited with exit code 1
2026-08-08 10:10:18.194 UTC [14435] LOG: shutting down
2026-08-08 10:10:18.372 UTC [14431] LOG: database system is shut down
2026-08-08T10:10:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53874 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:10:18 dist-test-slave-w206 krb5kdc[15998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:10:18 dist-test-slave-w206 krb5kdc[15998](info): set up 2 sockets
Aug 08 10:10:18 dist-test-slave-w206 krb5kdc[15998](info): commencing operation
krb5kdc: starting...
W20260808 10:10:19.620867 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:10:19 dist-test-slave-w206 krb5kdc[15998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:10:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:10:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:10:19.763477 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:43081
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:43081,127.2.30.189:41271,127.2.30.188:42841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:19.851994 16011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:19.852290 16011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:19.852334 16011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:19.853618 16011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:10:19.853722 16011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:19.853750 16011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:19.853775 16011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:10:19.853804 16011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:10:19.855553 16011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:43081,127.2.30.189:41271,127.2.30.188:42841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:43081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:19.855906 16011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:19.856180 16011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:19.862016 16016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:19.862143 16011 server_base.cc:1034] running on GCE node
W20260808 10:10:19.862027 16019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:19.862051 16017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:19.862672 16011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:19.863037 16011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:19.864216 16011 hybrid_clock.cc:648] HybridClock initialized: now 1786183819864205 us; error 36 us; skew 500 ppm
Aug 08 10:10:19 dist-test-slave-w206 krb5kdc[15998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183819, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:19.867128 16011 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:10:19.867712 16011 webserver.cc:460] Webserver started at http://127.2.30.190:45635/ using document root <none> and password file <none>
I20260808 10:10:19.867937 16011 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:19.867982 16011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:19.868083 16011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:10:19.869235 16011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "da987f00b7c143afa04a3a6977cec53e"
format_stamp: "Formatted at 2026-08-08 10:10:19 on dist-test-slave-w206"
server_key: "e086482804a24b8becfdd1a73d6c41a5"
server_key_iv: "a4ad03246d66c05bc3532cd42d09b1e3"
server_key_version: "encryptionkey@0"
I20260808 10:10:19.869556 16011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "da987f00b7c143afa04a3a6977cec53e"
format_stamp: "Formatted at 2026-08-08 10:10:19 on dist-test-slave-w206"
server_key: "e086482804a24b8becfdd1a73d6c41a5"
server_key_iv: "a4ad03246d66c05bc3532cd42d09b1e3"
server_key_version: "encryptionkey@0"
I20260808 10:10:19.871132 16011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:10:19.872059 16026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:19.872344 16011 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:19.872474 16011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "da987f00b7c143afa04a3a6977cec53e"
format_stamp: "Formatted at 2026-08-08 10:10:19 on dist-test-slave-w206"
server_key: "e086482804a24b8becfdd1a73d6c41a5"
server_key_iv: "a4ad03246d66c05bc3532cd42d09b1e3"
server_key_version: "encryptionkey@0"
I20260808 10:10:19.872558 16011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:19.883178 16011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:19.885746 16011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:19.885910 16011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:19.890637 16011 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:43081
I20260808 10:10:19.890707 16080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:43081 every 8 connection(s)
I20260808 10:10:19.891152 16011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:10:19.892315 16081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:19.894070 16081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:10:19.900856 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 16011
Aug 08 10:10:19 dist-test-slave-w206 krb5kdc[15998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183819, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:19.900993 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:10:19.901207 2170 external_mini_cluster.cc:1428] Setting key caac62022e8861a1c6d7fb8d17466b8f
I20260808 10:10:19.905400 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:19.894867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:43081 (local address 127.0.0.1:49754)
0808 10:10:19.895149 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:19.895198 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:19.895705 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:19.896008 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:19.896010 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:19.896319 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:19.896491 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:19.896496 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:19.897603 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:19.897604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:19.898343 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:19.898345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:19.898878 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:19.898879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:19.898945 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:19.899647 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:19.899655 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:19.901504 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:19.903647 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:19.903656 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:19.903658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:19.903953 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:19.904211 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:19.904212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:19.904212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:19.904346 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:19.904865 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:19.904870 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:19.904931 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 10:10:19.905007 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:10:19.905147 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":124,"threads_started":1}
I20260808 10:10:19.905421 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:19.894997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49754 (local address 127.2.30.190:43081)
0808 10:10:19.895161 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:19.895255 (+ 94us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:19.895291 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:10:19.895874 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:19.895880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:19.895900 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:19.896001 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:19.896616 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:19.897494 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:19.898454 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:19.898785 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:19.898920 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:19.901609 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:19.901618 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:19.901619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:19.901626 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:19.903543 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:19.904049 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:19.904051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:19.904051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:19.904135 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:19.904463 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:19.904464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:19.904465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:19.904790 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:19.904884 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:19.905066 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 10:10:19.905241 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260808 10:10:19.906543 16081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:10:19.907058 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:19.906866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:41271 (local address 127.0.0.1:50436)
0808 10:10:19.906938 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:19.907027 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:41271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 10:10:19.907203 16028 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.30.189:41271: connect: Connection refused (error 111)
I20260808 10:10:19.907780 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:19.907679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:41271 (local address 127.0.0.1:50450)
0808 10:10:19.907732 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:19.907758 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:41271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 10:10:19.907920 16081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.189:41271: Network error: Client connection negotiation failed: client connection to 127.2.30.189:41271: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.2.30.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.189.keytab.
Entry for principal kudu/127.2.30.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.189.keytab.
I20260808 10:10:19.936213 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.189:41271
--webserver_interface=127.2.30.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.189.keytab
--principal=kudu/127.2.30.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:43081,127.2.30.189:41271,127.2.30.188:42841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:10:19.957204 16081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } attempt: 1
I20260808 10:10:19.957836 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:19.957639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:41271 (local address 127.0.0.1:50462)
0808 10:10:19.957760 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:19.957803 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:41271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 10:10:19.958355 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:19.958207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:41271 (local address 127.0.0.1:50476)
0808 10:10:19.958289 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:19.958329 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:41271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 10:10:19.958495 16081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.189:41271: Network error: Client connection negotiation failed: client connection to 127.2.30.189:41271: connect: Connection refused (error 111)
W20260808 10:10:20.025409 16088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:20.025698 16088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:20.025741 16088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 10:10:20.026712 16081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } attempt: 2
W20260808 10:10:20.027092 16088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:10:20.027158 16088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:20.027190 16088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:20.027213 16088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:10:20.027242 16088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:10:20.027303 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.027125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:41271 (local address 127.0.0.1:50492)
0808 10:10:20.027198 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.027264 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:41271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 10:10:20.027706 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.027611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:41271 (local address 127.0.0.1:50504)
0808 10:10:20.027657 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.027682 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:41271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 10:10:20.027804 16081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.189:41271: Network error: Client connection negotiation failed: client connection to 127.2.30.189:41271: connect: Connection refused (error 111)
I20260808 10:10:20.029099 16088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:43081,127.2.30.189:41271,127.2.30.188:42841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.189.keytab
--principal=kudu/127.2.30.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.189:41271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.30.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:20.029413 16088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:20.029662 16088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:20.035248 16094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:20.035459 16095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:20.035449 16097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:20.035766 16088 server_base.cc:1034] running on GCE node
I20260808 10:10:20.036113 16088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:20.036521 16088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:20.037825 16088 hybrid_clock.cc:648] HybridClock initialized: now 1786183820037823 us; error 50 us; skew 500 ppm
Aug 08 10:10:20 dist-test-slave-w206 krb5kdc[15998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183820, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:20.040937 16088 init.cc:377] Logged in from keytab as kudu/127.2.30.189@KRBTEST.COM (short username kudu)
I20260808 10:10:20.041505 16088 webserver.cc:460] Webserver started at http://127.2.30.189:40907/ using document root <none> and password file <none>
I20260808 10:10:20.041714 16088 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:20.041759 16088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:20.041862 16088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:10:20.043140 16088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/data/instance:
uuid: "14f84d3626de4bf1802baf1824d0fdbc"
format_stamp: "Formatted at 2026-08-08 10:10:20 on dist-test-slave-w206"
server_key: "fe72affe2c260aaa833aa95ea0201b22"
server_key_iv: "7ea6a46a85fc2c1796df14c667c28d47"
server_key_version: "encryptionkey@0"
I20260808 10:10:20.043495 16088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/wal/instance:
uuid: "14f84d3626de4bf1802baf1824d0fdbc"
format_stamp: "Formatted at 2026-08-08 10:10:20 on dist-test-slave-w206"
server_key: "fe72affe2c260aaa833aa95ea0201b22"
server_key_iv: "7ea6a46a85fc2c1796df14c667c28d47"
server_key_version: "encryptionkey@0"
I20260808 10:10:20.044912 16088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:10:20.045814 16104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:20.046128 16088 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:20.046208 16088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/wal
uuid: "14f84d3626de4bf1802baf1824d0fdbc"
format_stamp: "Formatted at 2026-08-08 10:10:20 on dist-test-slave-w206"
server_key: "fe72affe2c260aaa833aa95ea0201b22"
server_key_iv: "7ea6a46a85fc2c1796df14c667c28d47"
server_key_version: "encryptionkey@0"
I20260808 10:10:20.046311 16088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:20.059929 16088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:20.062616 16088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:20.062753 16088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:20.067212 16088 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.189:41271
I20260808 10:10:20.067308 16157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.189:41271 every 8 connection(s)
I20260808 10:10:20.067684 16088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/data/info.pb
I20260808 10:10:20.068734 16158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:20.070425 16158 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:10:20.073557 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 16088
I20260808 10:10:20.073746 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-1/wal/instance
I20260808 10:10:20.074004 2170 external_mini_cluster.cc:1428] Setting key d45885d4060c2080a91083748a0a3108
Aug 08 10:10:20 dist-test-slave-w206 krb5kdc[15998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183820, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.189@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:20.082293 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.071348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49760 (local address 127.2.30.190:43081)
0808 10:10:20.071414 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:20.071417 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:20.071671 (+ 254us) server_negotiation.cc:407] Connection header received
0808 10:10:20.072266 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:20.072268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:20.072285 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:20.072438 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:20.073163 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.073736 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.075458 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.075677 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.075752 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.078338 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:20.078339 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:20.078340 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:20.078350 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:20.080435 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.080977 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.080979 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.080979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.081055 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.081476 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.081477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.081477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.081644 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:20.081722 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:20.082071 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 10:10:20.082176 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:10:20.082420 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.071316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:43081 (local address 127.0.0.1:49760)
0808 10:10:20.071535 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.071578 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:20.072145 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:20.072523 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:20.072527 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:20.072834 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:20.073055 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:20.073060 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.073883 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.073885 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:20.075297 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:20.075302 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.075789 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.075791 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:20.075992 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.076744 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:20.076753 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:20.078228 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:20.080554 (+ 2326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:20.080565 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:20.080568 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:20.080881 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:20.081165 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:20.081166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:20.081166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:20.081385 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:20.081761 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:20.081769 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:20.081915 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 10:10:20.082020 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 10:10:20.082167 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":93,"threads_started":1}
I20260808 10:10:20.083328 16158 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.30.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.188@KRBTEST.COM" created.
Aug 08 10:10:20 dist-test-slave-w206 krb5kdc[15998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183820, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.189@KRBTEST.COM for kudu/127.2.30.189@KRBTEST.COM
I20260808 10:10:20.092239 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.083765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50514 (local address 127.2.30.189:41271)
0808 10:10:20.084002 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:20.084006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:20.084026 (+ 20us) server_negotiation.cc:407] Connection header received
0808 10:10:20.084052 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:20.084054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:20.084065 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:20.084165 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:20.084483 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.085321 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.086218 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.086604 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.086701 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.088836 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:20.088849 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:20.088849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:20.088858 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:20.090713 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.091190 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.091193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.091194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.091256 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.091541 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.091541 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.091541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.091706 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:20.091798 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:20.091980 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 10:10:20.092116 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":196,"thread_start_us":138,"threads_started":1}
I20260808 10:10:20.092255 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.083666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:41271 (local address 127.0.0.1:50514)
0808 10:10:20.083812 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.083824 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:20.083874 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:20.084164 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:20.084165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:20.084342 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:20.084397 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:20.084398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.085438 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.085439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:20.086123 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:20.086126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.086700 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.086701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:20.086778 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.087398 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:20.087403 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:20.088708 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:20.090821 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:20.090823 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:20.090823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:20.091101 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:20.091380 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:20.091382 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:20.091382 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:20.091469 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:20.091778 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:20.091779 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:20.091840 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 10:10:20.091923 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 10:10:20.092058 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260808 10:10:20.093602 16158 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:10:20.094123 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.093963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:42841 (local address 127.0.0.1:51378)
0808 10:10:20.094026 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.094087 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 10:10:20.094362 16106 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
I20260808 10:10:20.095005 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.094851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:42841 (local address 127.0.0.1:51388)
0808 10:10:20.094911 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.094945 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 10:10:20.095117 16158 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:42841: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.188.keytab.
Entry for principal kudu/127.2.30.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.188.keytab.
I20260808 10:10:20.107930 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.188:42841
--webserver_interface=127.2.30.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.188.keytab
--principal=kudu/127.2.30.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:38359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:43081,127.2.30.189:41271,127.2.30.188:42841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:10:20.119016 16081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } attempt: 3
Aug 08 10:10:20 dist-test-slave-w206 krb5kdc[15998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183819, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for kudu/127.2.30.189@KRBTEST.COM
I20260808 10:10:20.127128 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.119486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:41271 (local address 127.0.0.1:50516)
0808 10:10:20.119552 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.119571 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:20.119652 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:20.119836 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:20.119837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:20.120036 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:20.120109 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:20.120111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.120854 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.120855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:20.121375 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:20.121376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.121786 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.121787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:20.121880 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.122443 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:20.122454 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:20.124111 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:20.125935 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:20.125937 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:20.125937 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:20.126128 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:20.126403 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:20.126404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:20.126405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:20.126473 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:20.126738 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:20.126739 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:20.126801 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 10:10:20.126887 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:10:20.127018 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 10:10:20.127128 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.119563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50516 (local address 127.2.30.189:41271)
0808 10:10:20.119639 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:20.119641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:20.119654 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:10:20.119728 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:20.119729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:20.119745 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:20.119828 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:20.120209 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.120734 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.121468 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.121694 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.121780 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.124223 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:20.124225 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:20.124225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:20.124236 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:20.125826 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.126207 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.126208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.126208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.126320 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.126540 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.126541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.126541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.126667 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:20.126748 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:20.126912 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 10:10:20.127029 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:10:20.127681 16081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:10:20.128079 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.127961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:42841 (local address 127.0.0.1:51392)
0808 10:10:20.128013 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.128043 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 10:10:20.128436 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.128360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:42841 (local address 127.0.0.1:51398)
0808 10:10:20.128395 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.128420 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260808 10:10:20.128544 16081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:42841: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
I20260808 10:10:20.144394 16158 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } attempt: 1
I20260808 10:10:20.145072 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.144906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:42841 (local address 127.0.0.1:51410)
0808 10:10:20.144976 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.145031 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 10:10:20.145530 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.145424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:42841 (local address 127.0.0.1:51412)
0808 10:10:20.145475 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.145507 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 10:10:20.145692 16158 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:42841: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
I20260808 10:10:20.159698 16081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } attempt: 1
I20260808 10:10:20.160259 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.160116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:42841 (local address 127.0.0.1:51420)
0808 10:10:20.160183 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.160226 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 10:10:20.160718 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.160615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:42841 (local address 127.0.0.1:51430)
0808 10:10:20.160668 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.160698 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 10:10:20.160848 16081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:42841: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
W20260808 10:10:20.200838 16166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:20.201138 16166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:20.201182 16166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:20.202917 16166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:10:20.203096 16166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:20.203119 16166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:20.203135 16166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:10:20.203147 16166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:10:20.204953 16166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:38359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:43081,127.2.30.189:41271,127.2.30.188:42841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.188.keytab
--principal=kudu/127.2.30.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.188:42841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.30.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:20.205255 16166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:20.205572 16166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:20.211314 16174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:20.211459 16171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:20.211459 16172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:20.211735 16166 server_base.cc:1034] running on GCE node
I20260808 10:10:20.212024 16166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:20.212450 16166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:20.213649 16166 hybrid_clock.cc:648] HybridClock initialized: now 1786183820213631 us; error 36 us; skew 500 ppm
I20260808 10:10:20.213848 16158 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } attempt: 2
I20260808 10:10:20.214454 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.214184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:42841 (local address 127.0.0.1:51432)
0808 10:10:20.214274 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.214408 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 10:10:20.214854 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.214784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:42841 (local address 127.0.0.1:51448)
0808 10:10:20.214811 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.214837 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260808 10:10:20.215008 16158 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:42841: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
Aug 08 10:10:20 dist-test-slave-w206 krb5kdc[15998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183820, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:20.216939 16166 init.cc:377] Logged in from keytab as kudu/127.2.30.188@KRBTEST.COM (short username kudu)
I20260808 10:10:20.217535 16166 webserver.cc:460] Webserver started at http://127.2.30.188:39669/ using document root <none> and password file <none>
I20260808 10:10:20.217914 16166 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:20.218021 16166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:20.218194 16166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:10:20.219528 16166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/data/instance:
uuid: "020c1cafa50040b0b85d2b7e2f093294"
format_stamp: "Formatted at 2026-08-08 10:10:20 on dist-test-slave-w206"
server_key: "5bacf378fa6bd9cd0d24456b0a751455"
server_key_iv: "6fc6f1a321ad19ddf89814e865000d2a"
server_key_version: "encryptionkey@0"
I20260808 10:10:20.219945 16166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/wal/instance:
uuid: "020c1cafa50040b0b85d2b7e2f093294"
format_stamp: "Formatted at 2026-08-08 10:10:20 on dist-test-slave-w206"
server_key: "5bacf378fa6bd9cd0d24456b0a751455"
server_key_iv: "6fc6f1a321ad19ddf89814e865000d2a"
server_key_version: "encryptionkey@0"
I20260808 10:10:20.221609 16166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 10:10:20.222675 16181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:20.223040 16166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:20.223169 16166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/wal
uuid: "020c1cafa50040b0b85d2b7e2f093294"
format_stamp: "Formatted at 2026-08-08 10:10:20 on dist-test-slave-w206"
server_key: "5bacf378fa6bd9cd0d24456b0a751455"
server_key_iv: "6fc6f1a321ad19ddf89814e865000d2a"
server_key_version: "encryptionkey@0"
I20260808 10:10:20.223266 16166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:20.235968 16081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } attempt: 2
I20260808 10:10:20.236699 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.236514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:42841 (local address 127.0.0.1:51456)
0808 10:10:20.236600 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.236655 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 10:10:20.238020 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.237305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:42841 (local address 127.0.0.1:51472)
0808 10:10:20.237943 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.237985 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":612}
W20260808 10:10:20.238137 16081 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:42841: Network error: Client connection negotiation failed: client connection to 127.2.30.188:42841: connect: Connection refused (error 111)
I20260808 10:10:20.248342 16166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:20.251116 16166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:20.251253 16166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:20.255832 16166 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.188:42841
I20260808 10:10:20.255916 16234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.188:42841 every 8 connection(s)
I20260808 10:10:20.256253 16166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/data/info.pb
I20260808 10:10:20.257283 16235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:20.259054 16235 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:10:20.265028 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 16166
I20260808 10:10:20.265184 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786183651681294-2170-0/minicluster-data/master-2/wal/instance
Aug 08 10:10:20 dist-test-slave-w206 krb5kdc[15998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183820, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.188@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:20.265601 2170 external_mini_cluster.cc:1428] Setting key 7186d952d041f3e7270e6f41205f3e7f
I20260808 10:10:20.269392 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.259910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49768 (local address 127.2.30.190:43081)
0808 10:10:20.259987 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:20.259989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:20.260243 (+ 254us) server_negotiation.cc:407] Connection header received
0808 10:10:20.260770 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:20.260771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:20.260782 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:20.260868 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:20.261542 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.262044 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.263177 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.263439 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.263503 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.266013 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:20.266016 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:20.266017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:20.266025 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:20.267745 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.268239 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.268240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.268241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.268310 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.268586 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.268587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.268588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.268733 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:20.268853 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:20.269160 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 10:10:20.269265 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:10:20.269552 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.259914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:43081 (local address 127.0.0.1:49768)
0808 10:10:20.260108 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.260150 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:20.260648 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:20.260954 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:20.260957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:20.261234 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:20.261448 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:20.261452 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.262140 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.262141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:20.263096 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:20.263100 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.263531 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.263532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:20.263652 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.264269 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:20.264278 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:20.265882 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:20.267827 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:20.267834 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:20.267836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:20.268141 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:20.268378 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:20.268379 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:20.268379 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:20.268521 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:20.268855 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:20.268860 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:20.268987 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 10:10:20.269076 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 10:10:20.269208 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":96,"threads_started":1}
I20260808 10:10:20.270437 16235 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:10:20 dist-test-slave-w206 krb5kdc[15998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.188@KRBTEST.COM
Aug 08 10:10:20 dist-test-slave-w206 krb5kdc[15998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183820, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.188@KRBTEST.COM for kudu/127.2.30.189@KRBTEST.COM
I20260808 10:10:20.276809 16239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.266094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51484 (local address 127.2.30.188:42841)
0808 10:10:20.266291 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:20.266298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:20.266405 (+ 107us) server_negotiation.cc:407] Connection header received
0808 10:10:20.266449 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:20.266452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:20.266469 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:20.266544 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:20.267403 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.268356 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.269310 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.269754 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.269844 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.272876 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:20.272885 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:20.272886 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:20.272897 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:20.275237 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.275656 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.275660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.275661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.275743 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.276045 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.276046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.276046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.276211 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:20.276411 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:20.276583 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 10:10:20.276658 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
I20260808 10:10:20.278510 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.270746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:41271 (local address 127.0.0.1:50522)
0808 10:10:20.270806 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.270818 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:20.270866 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:20.271059 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:20.271060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:20.271186 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:20.271238 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:20.271239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.272012 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.272013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:20.272638 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:20.272640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.273092 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.273093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:20.273177 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.273738 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:20.273743 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:20.275382 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:20.277267 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:20.277268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:20.277268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:20.277519 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:20.277755 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:20.277755 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:20.277756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:20.277818 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:20.278117 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:20.278118 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:20.278188 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 10:10:20.278291 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 10:10:20.278406 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 10:10:20.278517 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.270852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50522 (local address 127.2.30.189:41271)
0808 10:10:20.270910 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:20.270912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:20.270923 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:10:20.270951 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:20.270952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:20.270963 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:20.271056 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:20.271325 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.271930 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.272730 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.273002 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.273094 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.275483 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:20.275484 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:20.275485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:20.275493 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:20.277188 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.277601 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.277603 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.277603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.277673 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.277884 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.277885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.277886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.278031 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:20.278084 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:20.278341 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 10:10:20.278427 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 10:10:20.279142 16235 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:10:20 dist-test-slave-w206 krb5kdc[15998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
Aug 08 10:10:20 dist-test-slave-w206 krb5kdc[15998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183820, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.188@KRBTEST.COM for kudu/127.2.30.188@KRBTEST.COM
I20260808 10:10:20.286808 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.279446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:42841 (local address 127.0.0.1:51488)
0808 10:10:20.279500 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.279512 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:20.279564 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:20.279743 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:20.279744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:20.279908 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:20.279940 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:20.279941 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.280680 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.280681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:20.281380 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:20.281383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.281892 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.281892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:20.281944 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.282456 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:20.282461 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:20.283676 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:20.285433 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:20.285434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:20.285434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:20.285683 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:20.285920 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:20.285921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:20.285921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:20.286018 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:20.286377 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:20.286378 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:20.286413 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 10:10:20.286514 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 10:10:20.286649 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 10:10:20.286808 16239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.279508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51488 (local address 127.2.30.188:42841)
0808 10:10:20.279541 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:20.279543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:20.279552 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:10:20.279628 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:20.279628 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:20.279639 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:20.279739 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:20.280025 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.280567 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.281554 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.281800 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.281890 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.283763 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:20.283765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:20.283765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:20.283773 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:20.285332 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.285756 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.285757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.285758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.285815 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.286121 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.286122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.286123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.286273 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:20.286351 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:20.286568 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 10:10:20.286709 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":17}
I20260808 10:10:20.286994 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.277740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49782 (local address 127.2.30.190:43081)
0808 10:10:20.277784 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:20.277786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:20.278100 (+ 314us) server_negotiation.cc:407] Connection header received
0808 10:10:20.278126 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:20.278127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:20.278136 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:20.278197 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:20.278837 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.279565 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.280268 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.280485 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.280570 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.283091 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:20.283093 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:20.283094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:20.283101 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:20.284874 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.285244 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.285245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.285245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.285310 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.285618 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.285619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.285619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.285779 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:20.286716 (+ 937us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:20.286763 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 10:10:20.286856 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:10:20.288769 16235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294: Bootstrap starting.
I20260808 10:10:20.289819 16235 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:20.290190 16235 log.cc:824] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:20.291172 16235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294: No bootstrap required, opened a new log
Aug 08 10:10:20 dist-test-slave-w206 krb5kdc[15998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.189@KRBTEST.COM
I20260808 10:10:20.293207 16235 raft_consensus.cc:348] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } }
I20260808 10:10:20.293401 16235 raft_consensus.cc:374] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:20.293443 16235 raft_consensus.cc:729] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 020c1cafa50040b0b85d2b7e2f093294, State: Initialized, Role: FOLLOWER
I20260808 10:10:20.293599 16235 consensus_queue.cc:260] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } }
I20260808 10:10:20.294023 16235 sys_catalog.cc:564] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:10:20.294013 16241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } } }
I20260808 10:10:20.294129 16241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:10:20.296273 16255 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:10:20.296402 16255 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:10:20.296964 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.287940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50536 (local address 127.2.30.189:41271)
0808 10:10:20.287988 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:20.287990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:20.288023 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:10:20.288091 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:20.288092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:20.288103 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:20.288154 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:20.288621 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.289320 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.290045 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.290328 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.290410 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.293248 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:20.293250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:20.293250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:20.293260 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:20.295161 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.295587 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.295589 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.295590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.295661 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.295912 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.295913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.295914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.296074 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:20.296273 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:20.296772 (+ 499us) server_negotiation.cc:299] Negotiation successful
0808 10:10:20.296863 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:10:20.298446 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:10:20.296630 16166 master_runner.cc:186] Error getting master registration for 127.2.30.190:43081: OK, instance_id { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" instance_seqno: 1786183819889201 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:10:20.299371 16166 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:10:20.306131 16158 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } attempt: 3
I20260808 10:10:20.309664 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.300244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50550 (local address 127.2.30.189:41271)
0808 10:10:20.300330 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:20.300332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:20.300540 (+ 208us) server_negotiation.cc:407] Connection header received
0808 10:10:20.300650 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:20.300652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:20.300663 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:20.300694 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:20.301525 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.302280 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.303432 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.303751 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.303794 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.305768 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:20.305770 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:20.305770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:20.305778 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:20.307750 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.308190 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.308192 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.308192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.308251 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.308517 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.308518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.308518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.308667 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:20.309270 (+ 603us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:20.309476 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 10:10:20.309562 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 10:10:20.309710 16084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.300193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49794 (local address 127.2.30.190:43081)
0808 10:10:20.300293 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:20.300296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:20.300459 (+ 163us) server_negotiation.cc:407] Connection header received
0808 10:10:20.300573 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:20.300574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:20.300585 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:20.300616 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:20.301563 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.302285 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.303431 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.303739 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.303800 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.305894 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:20.305896 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:20.305896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:20.305904 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:20.308244 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.308649 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.308650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.308650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.308707 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.308910 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.308911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.308911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.309062 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:20.309160 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:20.309546 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 10:10:20.309620 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 10:10:20.310758 16239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.300687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51498 (local address 127.2.30.188:42841)
0808 10:10:20.300739 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:20.300741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:20.301148 (+ 407us) server_negotiation.cc:407] Connection header received
0808 10:10:20.301263 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:20.301264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:20.301275 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:20.301305 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:20.302099 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.302859 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.303907 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.304213 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.304257 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.305874 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:20.305876 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:20.305876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:20.305885 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:20.308528 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.309000 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.309002 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.309002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.309065 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.309290 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.309291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.309292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.309477 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:20.309558 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:20.310524 (+ 966us) server_negotiation.cc:299] Negotiation successful
0808 10:10:20.310644 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
Aug 08 10:10:20 dist-test-slave-w206 krb5kdc[15998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183820, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.189@KRBTEST.COM for kudu/127.2.30.188@KRBTEST.COM
I20260808 10:10:20.319341 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.306399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:42841 (local address 127.0.0.1:51510)
0808 10:10:20.306521 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.306533 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:20.306576 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:20.311030 (+ 4454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:20.311031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:20.311210 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:20.311258 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:20.311259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.312166 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.312167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:20.312784 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:20.312786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.313441 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.313442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:20.313514 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.314056 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:20.314060 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:20.315830 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:20.318070 (+ 2240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:20.318071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:20.318072 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:20.318266 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:20.318541 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:20.318542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:20.318542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:20.318610 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:20.318932 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:20.318933 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:20.319056 (+ 123us) client_negotiation.cc:769] Sending connection context
0808 10:10:20.319116 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 10:10:20.319267 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260808 10:10:20.319372 16239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.306655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51510 (local address 127.2.30.188:42841)
0808 10:10:20.310838 (+ 4183us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:20.310840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:20.310846 (+ 6us) server_negotiation.cc:407] Connection header received
0808 10:10:20.310862 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:20.310863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:20.310871 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:20.310994 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:20.311354 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.312083 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.312878 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.313196 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.313269 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.315950 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:20.315952 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:20.315952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:20.315960 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:20.317953 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.318382 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.318383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.318383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.318450 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.318683 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.318684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.318685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.318836 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:20.318908 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:20.319165 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 10:10:20.319264 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4169,"thread_start_us":141,"threads_started":1}
I20260808 10:10:20.321151 16158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc: Bootstrap starting.
I20260808 10:10:20.321970 16158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:20.322302 16158 log.cc:824] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:20.323362 16158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc: No bootstrap required, opened a new log
I20260808 10:10:20.325449 16158 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } }
I20260808 10:10:20.325594 16158 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:20.325631 16158 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14f84d3626de4bf1802baf1824d0fdbc, State: Initialized, Role: FOLLOWER
I20260808 10:10:20.325630 16241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:10:20.325742 16241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } }
I20260808 10:10:20.325801 16158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } }
I20260808 10:10:20.326114 16241 leader_election.cc:288] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da987f00b7c143afa04a3a6977cec53e (127.2.30.190:43081), 14f84d3626de4bf1802baf1824d0fdbc (127.2.30.189:41271)
I20260808 10:10:20.326149 16158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:10:20.326114 16266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } } }
I20260808 10:10:20.326354 16266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:10:20.326639 16132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "020c1cafa50040b0b85d2b7e2f093294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f84d3626de4bf1802baf1824d0fdbc" is_pre_election: true
I20260808 10:10:20.326764 16132 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 020c1cafa50040b0b85d2b7e2f093294 in term 0.
I20260808 10:10:20.327054 16183 leader_election.cc:302] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 020c1cafa50040b0b85d2b7e2f093294, 14f84d3626de4bf1802baf1824d0fdbc; no voters:
I20260808 10:10:20.327193 16241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:10:20.327243 16241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:10:20.327368 16241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:20.328428 16241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } }
I20260808 10:10:20.328574 16241 leader_election.cc:288] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [CANDIDATE]: Term 1 election: Requested vote from peers da987f00b7c143afa04a3a6977cec53e (127.2.30.190:43081), 14f84d3626de4bf1802baf1824d0fdbc (127.2.30.189:41271)
I20260808 10:10:20.329536 16132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "020c1cafa50040b0b85d2b7e2f093294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f84d3626de4bf1802baf1824d0fdbc"
I20260808 10:10:20.329610 16132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:20.330960 16132 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 020c1cafa50040b0b85d2b7e2f093294 in term 1.
I20260808 10:10:20.331173 16183 leader_election.cc:302] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 020c1cafa50040b0b85d2b7e2f093294, 14f84d3626de4bf1802baf1824d0fdbc; no voters:
I20260808 10:10:20.331284 16241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:10:20.331440 16241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [term 1 LEADER]: Becoming Leader. State: Replica: 020c1cafa50040b0b85d2b7e2f093294, State: Running, Role: LEADER
I20260808 10:10:20.328909 16088 master_runner.cc:186] Error getting master registration for 127.2.30.190:43081: OK, instance_id { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" instance_seqno: 1786183819889201 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:10:20.331593 16088 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:10:20.331549 16241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } }
I20260808 10:10:20.331995 16241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 020c1cafa50040b0b85d2b7e2f093294. Latest consensus state: current_term: 1 leader_uuid: "020c1cafa50040b0b85d2b7e2f093294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } } }
I20260808 10:10:20.332084 16241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [sys.catalog]: This master's current role is: LEADER
W20260808 10:10:20.332291 16277 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:10:20.332412 16277 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:10:20.337250 16081 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } attempt: 3
I20260808 10:10:20.339107 16281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:10:20.339356 16281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:10:20.341575 16132 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc [term 1 FOLLOWER]: Refusing update from remote peer 020c1cafa50040b0b85d2b7e2f093294: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
Aug 08 10:10:20 dist-test-slave-w206 krb5kdc[15998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183819, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for kudu/127.2.30.188@KRBTEST.COM
I20260808 10:10:20.343029 16241 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:10:20.345127 16266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 020c1cafa50040b0b85d2b7e2f093294. Latest consensus state: current_term: 1 leader_uuid: "020c1cafa50040b0b85d2b7e2f093294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } } }
I20260808 10:10:20.345276 16266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:10:20.348110 16282 mvcc.cc:205] Tried to move back new op lower bound from 7316208928115527680 to 7316208928078852096. Current Snapshot: MvccSnapshot[applied={T|T < 7316208928115527680}]
I20260808 10:10:20.348692 16241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "020c1cafa50040b0b85d2b7e2f093294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } } }
I20260808 10:10:20.348820 16241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [sys.catalog]: This master's current role is: LEADER
I20260808 10:10:20.349419 16266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "020c1cafa50040b0b85d2b7e2f093294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } } }
I20260808 10:10:20.349522 16266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:10:20.349972 16283 mvcc.cc:205] Tried to move back new op lower bound from 7316208928115527680 to 7316208928078852096. Current Snapshot: MvccSnapshot[applied={T|T < 7316208928115527680}]
I20260808 10:10:20.350193 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.337577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:42841 (local address 127.0.0.1:51514)
0808 10:10:20.337632 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:20.337644 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:20.337685 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:20.337988 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:20.337989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:20.338199 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:20.338256 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:20.338259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.339298 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.339300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:20.340151 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:20.340155 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.340769 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.340770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:20.340847 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.341460 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:20.341466 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:20.345848 (+ 4382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:20.348421 (+ 2573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:20.348423 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:20.348423 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:20.348703 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:20.349177 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:20.349178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:20.349178 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:20.349263 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:20.349823 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:20.349824 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:20.349898 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 10:10:20.349967 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 10:10:20.350085 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 10:10:20.350886 16239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:20.337802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51514 (local address 127.2.30.188:42841)
0808 10:10:20.337860 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:20.337862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:20.337871 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:10:20.337896 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:20.337897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:20.337908 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:20.337973 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:20.338364 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.339158 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.340262 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:20.340569 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:20.340638 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:20.345939 (+ 5301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:20.345940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:20.345941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:20.345950 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:20.348290 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.348999 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.349001 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.349001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.349085 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:20.349381 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:20.349382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:20.349382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:20.349544 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:20.349606 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:20.350697 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0808 10:10:20.350792 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:10:20.352068 16281 catalog_manager.cc:1347] Generated new cluster ID: 3664462689aa41a3ae7d034a3be3190a
I20260808 10:10:20.352128 16281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:10:20.352676 16081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e: Bootstrap starting.
I20260808 10:10:20.353433 16081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:20.353734 16081 log.cc:824] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:20.354739 16081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e: No bootstrap required, opened a new log
I20260808 10:10:20.356628 16081 raft_consensus.cc:348] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } }
I20260808 10:10:20.356786 16081 raft_consensus.cc:374] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:20.356817 16081 raft_consensus.cc:729] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da987f00b7c143afa04a3a6977cec53e, State: Initialized, Role: FOLLOWER
I20260808 10:10:20.356976 16081 consensus_queue.cc:260] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } }
I20260808 10:10:20.357415 16289 sys_catalog.cc:455] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } } }
I20260808 10:10:20.357544 16053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "020c1cafa50040b0b85d2b7e2f093294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da987f00b7c143afa04a3a6977cec53e" is_pre_election: true
I20260808 10:10:20.357616 16052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "020c1cafa50040b0b85d2b7e2f093294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da987f00b7c143afa04a3a6977cec53e"
I20260808 10:10:20.357591 16051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:20.357640 16289 sys_catalog.cc:458] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:10:20.358786 16051 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e [term 1 FOLLOWER]: Refusing update from remote peer 020c1cafa50040b0b85d2b7e2f093294: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:10:20.357512 16081 sys_catalog.cc:564] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:10:20.359165 16286 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 10:10:20.361428 16303 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:10:20.361487 16303 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:10:20.360752 16289 sys_catalog.cc:455] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 020c1cafa50040b0b85d2b7e2f093294. Latest consensus state: current_term: 1 leader_uuid: "020c1cafa50040b0b85d2b7e2f093294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } } }
I20260808 10:10:20.361675 16289 sys_catalog.cc:458] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:10:20.360886 16302 sys_catalog.cc:455] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "020c1cafa50040b0b85d2b7e2f093294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } } }
I20260808 10:10:20.364017 16302 sys_catalog.cc:458] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:10:20.364655 16297 mvcc.cc:205] Tried to move back new op lower bound from 7316208928115527680 to 7316208928078852096. Current Snapshot: MvccSnapshot[applied={T|T < 7316208928115527680}]
I20260808 10:10:20.387283 16278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "020c1cafa50040b0b85d2b7e2f093294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da987f00b7c143afa04a3a6977cec53e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43081 } } peers { permanent_uuid: "14f84d3626de4bf1802baf1824d0fdbc" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 41271 } } peers { permanent_uuid: "020c1cafa50040b0b85d2b7e2f093294" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 42841 } } }
I20260808 10:10:20.387455 16278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294 [sys.catalog]: This master's current role is: LEADER
I20260808 10:10:20.389976 16281 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:10:20.390921 16281 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:10:20.397670 16281 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 020c1cafa50040b0b85d2b7e2f093294: Generated new TSK 0
I20260808 10:10:20.397990 16281 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:10:21.332986 16277 catalog_manager.cc:1259] Loaded cluster ID: 3664462689aa41a3ae7d034a3be3190a
I20260808 10:10:21.333073 16277 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc: loading cluster ID for follower catalog manager: success
I20260808 10:10:21.334024 16277 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc: acquiring CA information for follower catalog manager: success
I20260808 10:10:21.334322 16277 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 14f84d3626de4bf1802baf1824d0fdbc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:10:21.361953 16303 catalog_manager.cc:1259] Loaded cluster ID: 3664462689aa41a3ae7d034a3be3190a
I20260808 10:10:21.362037 16303 catalog_manager.cc:1552] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e: loading cluster ID for follower catalog manager: success
I20260808 10:10:21.362812 16303 catalog_manager.cc:1574] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e: acquiring CA information for follower catalog manager: success
I20260808 10:10:21.363176 16303 catalog_manager.cc:1602] T 00000000000000000000000000000000 P da987f00b7c143afa04a3a6977cec53e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:10:21.461256 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 16011
I20260808 10:10:21.468698 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 16088
I20260808 10:10:21.475726 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 16166
2026-08-08T10:10:21Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2969 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:10:21 dist-test-slave-w206 krb5kdc[16313](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:10:21 dist-test-slave-w206 krb5kdc[16313](info): set up 2 sockets
Aug 08 10:10:21 dist-test-slave-w206 krb5kdc[16313](info): commencing operation
krb5kdc: starting...
W20260808 10:10:22.606572 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:10:22 dist-test-slave-w206 krb5kdc[16313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:10:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:10:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:10:22.748195 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:37147
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:37147
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:22.836577 16326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:22.836854 16326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:22.836894 16326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:22.838085 16326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:10:22.838164 16326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:22.838188 16326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:22.838208 16326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:10:22.838264 16326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:10:22.839875 16326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:37147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:37147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:22.840198 16326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:22.840454 16326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:22.845963 16333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:22.846040 16331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:22.846174 16336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:22.846670 16326 server_base.cc:1034] running on GCE node
I20260808 10:10:22.846948 16326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:22.847357 16326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:22.848556 16326 hybrid_clock.cc:648] HybridClock initialized: now 1786183822848540 us; error 44 us; skew 500 ppm
Aug 08 10:10:22 dist-test-slave-w206 krb5kdc[16313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183822, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:22.851675 16326 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:10:22.852236 16326 webserver.cc:460] Webserver started at http://127.2.30.190:44445/ using document root <none> and password file <none>
I20260808 10:10:22.852528 16326 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:22.852614 16326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:22.852798 16326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:10:22.854126 16326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "6d445bfddfd94cae80fa267ca102ab79"
format_stamp: "Formatted at 2026-08-08 10:10:22 on dist-test-slave-w206"
server_key: "454942d5fe6b0697346e9784fc38f054"
server_key_iv: "5ba370b56e9f2eba6f10f6c58cedbbb7"
server_key_version: "encryptionkey@0"
I20260808 10:10:22.854720 16326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "6d445bfddfd94cae80fa267ca102ab79"
format_stamp: "Formatted at 2026-08-08 10:10:22 on dist-test-slave-w206"
server_key: "454942d5fe6b0697346e9784fc38f054"
server_key_iv: "5ba370b56e9f2eba6f10f6c58cedbbb7"
server_key_version: "encryptionkey@0"
I20260808 10:10:22.856401 16326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 10:10:22.857285 16343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:22.857513 16326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:22.857631 16326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "6d445bfddfd94cae80fa267ca102ab79"
format_stamp: "Formatted at 2026-08-08 10:10:22 on dist-test-slave-w206"
server_key: "454942d5fe6b0697346e9784fc38f054"
server_key_iv: "5ba370b56e9f2eba6f10f6c58cedbbb7"
server_key_version: "encryptionkey@0"
I20260808 10:10:22.857714 16326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:22.864426 16326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:22.866863 16326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:22.867077 16326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:22.871577 16326 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:37147
I20260808 10:10:22.871783 16397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:37147 every 8 connection(s)
I20260808 10:10:22.872031 16326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:10:22.873081 16398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:22.875787 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 16326
I20260808 10:10:22.875922 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:10:22.876137 2170 external_mini_cluster.cc:1428] Setting key 6f6368ffd4412cbd1e44bdaed612da7e
I20260808 10:10:22.876183 16398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79: Bootstrap starting.
I20260808 10:10:22.877054 16398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:22.877391 16398 log.cc:824] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:22.878144 16398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79: No bootstrap required, opened a new log
I20260808 10:10:22.880549 16398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d445bfddfd94cae80fa267ca102ab79" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37147 } }
I20260808 10:10:22.880712 16398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:22.880759 16398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d445bfddfd94cae80fa267ca102ab79, State: Initialized, Role: FOLLOWER
I20260808 10:10:22.880925 16398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d445bfddfd94cae80fa267ca102ab79" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37147 } }
I20260808 10:10:22.881038 16398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:10:22.881073 16398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:10:22.881111 16398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:22.881914 16398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d445bfddfd94cae80fa267ca102ab79" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37147 } }
I20260808 10:10:22.882032 16398 leader_election.cc:302] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d445bfddfd94cae80fa267ca102ab79; no voters:
I20260808 10:10:22.882220 16398 leader_election.cc:288] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:10:22.882354 16403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:10:22.882495 16398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:10:22.882529 16403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [term 1 LEADER]: Becoming Leader. State: Replica: 6d445bfddfd94cae80fa267ca102ab79, State: Running, Role: LEADER
I20260808 10:10:22.882648 16403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d445bfddfd94cae80fa267ca102ab79" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37147 } }
I20260808 10:10:22.883045 16403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d445bfddfd94cae80fa267ca102ab79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d445bfddfd94cae80fa267ca102ab79" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37147 } } }
I20260808 10:10:22.883136 16403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [sys.catalog]: This master's current role is: LEADER
I20260808 10:10:22.883312 16403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d445bfddfd94cae80fa267ca102ab79. Latest consensus state: current_term: 1 leader_uuid: "6d445bfddfd94cae80fa267ca102ab79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d445bfddfd94cae80fa267ca102ab79" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37147 } } }
I20260808 10:10:22.883383 16403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79 [sys.catalog]: This master's current role is: LEADER
Aug 08 10:10:22 dist-test-slave-w206 krb5kdc[16313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
W20260808 10:10:22.884745 16418 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:10:22.884805 16418 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:10:22.885205 16412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:10:22.885434 16412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:10:22.887040 16412 catalog_manager.cc:1347] Generated new cluster ID: 86e92844fa2241efab2cd65ef126132f
I20260808 10:10:22.887120 16412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:10:22.889312 16401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:22.876727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49570 (local address 127.2.30.190:37147)
0808 10:10:22.876935 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:22.876943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:22.876970 (+ 27us) server_negotiation.cc:407] Connection header received
0808 10:10:22.877558 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:22.877566 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:22.877618 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:22.877738 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:22.878553 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:22.880030 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:22.880989 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:22.881626 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:22.881745 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:22.885161 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:22.885165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:22.885165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:22.885177 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:22.887539 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:22.887959 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:22.887964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:22.887964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:22.888080 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:22.888359 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:22.888360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:22.888361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:22.888673 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:22.888839 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:22.888920 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 10:10:22.889063 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":106,"threads_started":1}
I20260808 10:10:22.890232 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:10:22.892670 16412 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:10:22.893373 16412 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:10:22.900981 16412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d445bfddfd94cae80fa267ca102ab79: Generated new TSK 0
I20260808 10:10:22.901177 16412 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:10:22.945235 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 16326
2026-08-08T10:10:22Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1468 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:10:23 dist-test-slave-w206 krb5kdc[16425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:10:23 dist-test-slave-w206 krb5kdc[16425](info): set up 2 sockets
Aug 08 10:10:23 dist-test-slave-w206 krb5kdc[16425](info): commencing operation
krb5kdc: starting...
W20260808 10:10:24.084941 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:10:24 dist-test-slave-w206 krb5kdc[16425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:10:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:10:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:10:24.244007 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:36255
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:36255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:24.334897 16438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:24.335246 16438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:24.335381 16438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:24.336858 16438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:10:24.336920 16438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:24.336936 16438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:24.336949 16438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:10:24.336962 16438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:10:24.338867 16438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:36255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:36255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:24.339286 16438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:24.339890 16438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:24.345901 16444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:24.345894 16446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:24.345894 16443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:24.346606 16438 server_base.cc:1034] running on GCE node
I20260808 10:10:24.347079 16438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:24.347450 16438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:24.348611 16438 hybrid_clock.cc:648] HybridClock initialized: now 1786183824348571 us; error 62 us; skew 500 ppm
Aug 08 10:10:24 dist-test-slave-w206 krb5kdc[16425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183824, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:24.352072 16438 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:10:24.352808 16438 webserver.cc:460] Webserver started at http://127.2.30.190:44065/ using document root <none> and password file <none>
I20260808 10:10:24.353080 16438 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:24.353178 16438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:24.353344 16438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:10:24.354559 16438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "27a09832204248caac5ee1bc22caa102"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "31b21137f1272ab7ccf0aabc257a3dbb"
server_key_iv: "b2e3062134e1c25ef0a257dba199d82c"
server_key_version: "encryptionkey@0"
I20260808 10:10:24.355037 16438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "27a09832204248caac5ee1bc22caa102"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "31b21137f1272ab7ccf0aabc257a3dbb"
server_key_iv: "b2e3062134e1c25ef0a257dba199d82c"
server_key_version: "encryptionkey@0"
I20260808 10:10:24.356734 16438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:10:24.357875 16453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:24.358254 16438 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:10:24.358399 16438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "27a09832204248caac5ee1bc22caa102"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "31b21137f1272ab7ccf0aabc257a3dbb"
server_key_iv: "b2e3062134e1c25ef0a257dba199d82c"
server_key_version: "encryptionkey@0"
I20260808 10:10:24.358522 16438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:24.385473 16438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:24.388907 16438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:24.389149 16438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:24.394237 16438 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:36255
I20260808 10:10:24.394277 16507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:36255 every 8 connection(s)
I20260808 10:10:24.394742 16438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:10:24.395994 16508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:24.398756 16508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102: Bootstrap starting.
I20260808 10:10:24.399505 16508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:24.399840 16508 log.cc:824] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:24.401197 16508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102: No bootstrap required, opened a new log
I20260808 10:10:24.401811 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 16438
I20260808 10:10:24.401957 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:10:24.402237 2170 external_mini_cluster.cc:1428] Setting key 1b983b1ddb0d009de6da80960f501791
I20260808 10:10:24.404379 16508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } }
I20260808 10:10:24.404621 16508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:24.404664 16508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a09832204248caac5ee1bc22caa102, State: Initialized, Role: FOLLOWER
I20260808 10:10:24.404821 16508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } }
I20260808 10:10:24.404935 16508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:10:24.404973 16508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:10:24.405028 16508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:24.405776 16508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } }
I20260808 10:10:24.405877 16508 leader_election.cc:302] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27a09832204248caac5ee1bc22caa102; no voters:
I20260808 10:10:24.406132 16508 leader_election.cc:288] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:10:24.406183 16513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:10:24.406327 16513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 1 LEADER]: Becoming Leader. State: Replica: 27a09832204248caac5ee1bc22caa102, State: Running, Role: LEADER
I20260808 10:10:24.406415 16513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } }
I20260808 10:10:24.406450 16508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:10:24.406947 16513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27a09832204248caac5ee1bc22caa102. Latest consensus state: current_term: 1 leader_uuid: "27a09832204248caac5ee1bc22caa102" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } } }
I20260808 10:10:24.407056 16513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [sys.catalog]: This master's current role is: LEADER
I20260808 10:10:24.407246 16515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27a09832204248caac5ee1bc22caa102" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } } }
I20260808 10:10:24.407326 16515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [sys.catalog]: This master's current role is: LEADER
I20260808 10:10:24.407478 16518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:10:24.408071 16518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:10:24.410059 16518 catalog_manager.cc:1347] Generated new cluster ID: fbbbfb597ed84ddda1734996d00f91d6
I20260808 10:10:24.410128 16518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:10:24 dist-test-slave-w206 krb5kdc[16425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:24.416721 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:24.402924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46856 (local address 127.2.30.190:36255)
0808 10:10:24.403192 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:24.403201 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:24.403232 (+ 31us) server_negotiation.cc:407] Connection header received
0808 10:10:24.404151 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:24.404160 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:24.404223 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:24.404354 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:24.405257 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:24.406463 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:24.407598 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:24.408287 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:24.408784 (+ 497us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:24.412420 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:24.412428 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:24.412428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:24.412445 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:24.414812 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:24.415306 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:24.415310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:24.415311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:24.415423 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:24.415659 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:24.415660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:24.415660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:24.415952 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:24.416135 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:24.416272 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 10:10:24.416538 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":174,"threads_started":1}
I20260808 10:10:24.433212 16518 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:10:24.434048 16518 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:10:24.439445 16518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102: Generated new TSK 0
I20260808 10:10:24.439726 16518 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:10:24.486716 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36255
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:24.588215 16536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:24.588495 16536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:24.588526 16536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:24.590008 16536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:24.590080 16536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:24.590113 16536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:10:24.592115 16536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:24.592442 16536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:24.592770 16536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:24.598958 16542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:24.598934 16541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:24.598928 16544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:24.599196 16536 server_base.cc:1034] running on GCE node
I20260808 10:10:24.599495 16536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:24.599792 16536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:24.601058 16536 hybrid_clock.cc:648] HybridClock initialized: now 1786183824601061 us; error 52 us; skew 500 ppm
Aug 08 10:10:24 dist-test-slave-w206 krb5kdc[16425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183824, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:24.604503 16536 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:10:24.605091 16536 webserver.cc:460] Webserver started at http://127.2.30.129:45453/ using document root <none> and password file <none>
I20260808 10:10:24.605376 16536 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:24.605453 16536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:24.605618 16536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:10:24.606869 16536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "bdf56168db55404981e1183fb713760c"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "4ad3568d7c8112d3df672f1ae24735c9"
server_key_iv: "1c3918e6fdaff38a09d6b4d6c9f04355"
server_key_version: "encryptionkey@0"
I20260808 10:10:24.607332 16536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "bdf56168db55404981e1183fb713760c"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "4ad3568d7c8112d3df672f1ae24735c9"
server_key_iv: "1c3918e6fdaff38a09d6b4d6c9f04355"
server_key_version: "encryptionkey@0"
I20260808 10:10:24.608956 16536 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:10:24.610060 16551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:24.610482 16536 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:24.610646 16536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "bdf56168db55404981e1183fb713760c"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "4ad3568d7c8112d3df672f1ae24735c9"
server_key_iv: "1c3918e6fdaff38a09d6b4d6c9f04355"
server_key_version: "encryptionkey@0"
I20260808 10:10:24.610749 16536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:24.628417 16536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:24.631206 16536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:24.631402 16536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:24.631696 16536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:24.632121 16536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:10:24.632184 16536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:24.632231 16536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:10:24.632263 16536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:24.639307 16536 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:33175
I20260808 10:10:24.639456 16668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:33175 every 8 connection(s)
I20260808 10:10:24.639775 16536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:10:24.644685 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 16536
I20260808 10:10:24.644863 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:10:24.645069 2170 external_mini_cluster.cc:1428] Setting key 60f97ca756ab38f9f54d0530c86d1fe3
Aug 08 10:10:24 dist-test-slave-w206 krb5kdc[16425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183824, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:24.651156 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:24.640691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:34757 (local address 127.2.30.190:36255)
0808 10:10:24.640771 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:24.640774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:24.640973 (+ 199us) server_negotiation.cc:407] Connection header received
0808 10:10:24.641566 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:24.641568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:24.641582 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:24.641648 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:24.642738 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:24.643357 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:24.644472 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:24.644681 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:24.644764 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:24.647615 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:24.647616 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:24.647617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:24.647626 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:24.649524 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:24.650020 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:24.650021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:24.650022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:24.650092 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:24.650393 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:24.650394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:24.650395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:24.650531 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:24.650608 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:24.650921 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 10:10:24.651050 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:10:24.651400 16671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:24.640556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:34757)
0808 10:10:24.640821 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:24.640879 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:24.641446 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:24.641729 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:24.641733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:24.642389 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:24.642623 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:24.642629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:24.643478 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:24.643479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:24.644341 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:24.644344 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:24.644780 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:24.644781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:24.644978 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:24.645708 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:24.645719 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:24.647499 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:24.649626 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:24.649638 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:24.649641 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:24.649921 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:24.650178 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:24.650179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:24.650179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:24.650311 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:24.650607 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:24.650616 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:24.650778 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 10:10:24.650900 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 10:10:24.651124 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":104,"threads_started":1}
I20260808 10:10:24.652251 16669 heartbeater.cc:347] Connected to a master server at 127.2.30.190:36255
I20260808 10:10:24.652382 16669 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:24.652602 16669 heartbeater.cc:511] Master 127.2.30.190:36255 requested a full tablet report, sending...
I20260808 10:10:24.653321 16470 ts_manager.cc:194] Registered new tserver with Master: bdf56168db55404981e1183fb713760c (127.2.30.129:33175)
I20260808 10:10:24.654042 16470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:34757
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:10:24.679816 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36255
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:24.778697 16675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:24.779035 16675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:24.779093 16675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:24.780565 16675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:24.780653 16675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:24.780705 16675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:10:24.782444 16675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:24.782835 16675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:24.783180 16675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:24.789004 16680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:24.789175 16675 server_base.cc:1034] running on GCE node
W20260808 10:10:24.789258 16684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:24.789047 16681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:24.789543 16675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:24.789841 16675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:24.791091 16675 hybrid_clock.cc:648] HybridClock initialized: now 1786183824791050 us; error 59 us; skew 500 ppm
Aug 08 10:10:24 dist-test-slave-w206 krb5kdc[16425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183824, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:24.794100 16675 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:10:24.794688 16675 webserver.cc:460] Webserver started at http://127.2.30.130:44741/ using document root <none> and password file <none>
I20260808 10:10:24.795042 16675 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:24.795099 16675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:24.795200 16675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:10:24.796258 16675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "c8b8c6016b1648cab00c272f178a0137"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "3a7035de113beda7dde2c15de00f50cf"
server_key_iv: "2e1b074a7cdb6b59915a01e0fb6ad080"
server_key_version: "encryptionkey@0"
I20260808 10:10:24.796624 16675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "c8b8c6016b1648cab00c272f178a0137"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "3a7035de113beda7dde2c15de00f50cf"
server_key_iv: "2e1b074a7cdb6b59915a01e0fb6ad080"
server_key_version: "encryptionkey@0"
I20260808 10:10:24.798319 16675 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:10:24.799372 16691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:24.799587 16675 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:24.799685 16675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "c8b8c6016b1648cab00c272f178a0137"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "3a7035de113beda7dde2c15de00f50cf"
server_key_iv: "2e1b074a7cdb6b59915a01e0fb6ad080"
server_key_version: "encryptionkey@0"
I20260808 10:10:24.799758 16675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:24.813935 16675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:24.816561 16675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:24.816690 16675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:24.816906 16675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:24.817273 16675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:10:24.817320 16675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:24.817344 16675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:10:24.817394 16675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:24.823484 16675 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:35479
I20260808 10:10:24.823585 16806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:35479 every 8 connection(s)
I20260808 10:10:24.823927 16675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:10:24.827214 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 16675
I20260808 10:10:24.827353 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:10:24.827545 2170 external_mini_cluster.cc:1428] Setting key 105a1ff43b11c78df7c8eb77ca257ae5
Aug 08 10:10:24 dist-test-slave-w206 krb5kdc[16425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183824, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:24.836429 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:24.824995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:51973 (local address 127.2.30.190:36255)
0808 10:10:24.825091 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:24.825093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:24.825225 (+ 132us) server_negotiation.cc:407] Connection header received
0808 10:10:24.825772 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:24.825773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:24.825787 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:24.825838 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:24.827049 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:24.827798 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:24.829389 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:24.829755 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:24.829858 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:24.832974 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:24.832976 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:24.832976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:24.832993 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:24.834685 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:24.835289 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:24.835291 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:24.835292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:24.835377 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:24.835679 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:24.835680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:24.835681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:24.835818 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:24.835903 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:24.836231 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 10:10:24.836322 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 10:10:24.836661 16809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:24.824907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:51973)
0808 10:10:24.825091 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:24.825133 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:24.825650 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:24.825914 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:24.825918 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:24.826584 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:24.826853 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:24.826860 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:24.827947 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:24.827948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:24.829019 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:24.829023 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:24.829888 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:24.829890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:24.830027 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:24.830720 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:24.830733 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:24.832828 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:24.834815 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:24.834829 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:24.834832 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:24.835198 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:24.835458 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:24.835459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:24.835460 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:24.835600 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:24.835905 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:24.835914 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:24.836081 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 10:10:24.836195 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 10:10:24.836397 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
I20260808 10:10:24.837231 16807 heartbeater.cc:347] Connected to a master server at 127.2.30.190:36255
I20260808 10:10:24.837378 16807 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:24.837631 16807 heartbeater.cc:511] Master 127.2.30.190:36255 requested a full tablet report, sending...
I20260808 10:10:24.838271 16470 ts_manager.cc:194] Registered new tserver with Master: c8b8c6016b1648cab00c272f178a0137 (127.2.30.130:35479)
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
I20260808 10:10:24.838850 16470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:51973
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:10:24.863914 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36255
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:24.963596 16812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:24.963878 16812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:24.963924 16812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:24.965221 16812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:24.965306 16812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:24.965358 16812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:10:24.967684 16812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:24.968030 16812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:24.968298 16812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:24.974161 16818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:24.974185 16821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:24.974464 16812 server_base.cc:1034] running on GCE node
W20260808 10:10:24.974181 16819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:24.974800 16812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:24.975135 16812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:24.976389 16812 hybrid_clock.cc:648] HybridClock initialized: now 1786183824976377 us; error 43 us; skew 500 ppm
Aug 08 10:10:24 dist-test-slave-w206 krb5kdc[16425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183824, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:24.979344 16812 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:10:24.979825 16812 webserver.cc:460] Webserver started at http://127.2.30.131:34205/ using document root <none> and password file <none>
I20260808 10:10:24.980111 16812 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:24.980163 16812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:24.980309 16812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:10:24.981518 16812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "e586a293ae614b3f8a1b77faf45c331b"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "1f3377ed638db8ed602b1fb38b643ed9"
server_key_iv: "773ff875570ce051703126f6272d074f"
server_key_version: "encryptionkey@0"
I20260808 10:10:24.981890 16812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "e586a293ae614b3f8a1b77faf45c331b"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "1f3377ed638db8ed602b1fb38b643ed9"
server_key_iv: "773ff875570ce051703126f6272d074f"
server_key_version: "encryptionkey@0"
I20260808 10:10:24.983549 16812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:10:24.984848 16828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:24.985134 16812 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:24.985261 16812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "e586a293ae614b3f8a1b77faf45c331b"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "1f3377ed638db8ed602b1fb38b643ed9"
server_key_iv: "773ff875570ce051703126f6272d074f"
server_key_version: "encryptionkey@0"
I20260808 10:10:24.985378 16812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:25.015501 16812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:25.018369 16812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:25.018585 16812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:25.018882 16812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:25.019273 16812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:10:25.019336 16812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:25.019374 16812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:10:25.019405 16812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:25.026206 16812 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:37833
I20260808 10:10:25.026249 16944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:37833 every 8 connection(s)
I20260808 10:10:25.026672 16812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:10:25.031901 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 16812
I20260808 10:10:25.032014 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:10:25.032178 2170 external_mini_cluster.cc:1428] Setting key 35195dc749a792c74a013599a14e14f3
Aug 08 10:10:25 dist-test-slave-w206 krb5kdc[16425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183824, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:25.038148 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.027650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:50681 (local address 127.2.30.190:36255)
0808 10:10:25.027735 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.027738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.027994 (+ 256us) server_negotiation.cc:407] Connection header received
0808 10:10:25.028607 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.028608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.028624 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.028677 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:25.029752 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.030296 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.031614 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.031924 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.032014 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.034963 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:25.034965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:25.034965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:25.035021 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:25.036676 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:25.037158 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:25.037160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:25.037160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:25.037223 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:25.037504 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:25.037505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:25.037505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:25.037646 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:25.037714 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.037982 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.038062 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:10:25.038412 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.027621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:50681)
0808 10:10:25.027855 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:25.027898 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:25.028464 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:25.028786 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:25.028790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:25.029421 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:25.029629 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.029634 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.030493 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.030494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.031494 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.031498 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.032067 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.032069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.032406 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.033162 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:25.033174 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:25.034853 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:25.036777 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:25.036790 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:25.036793 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:25.037059 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:25.037303 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:25.037304 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:25.037304 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:25.037419 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:25.037716 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:25.037721 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:25.037860 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 10:10:25.037948 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:10:25.038089 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":114,"threads_started":1}
I20260808 10:10:25.039093 16945 heartbeater.cc:347] Connected to a master server at 127.2.30.190:36255
I20260808 10:10:25.039223 16945 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:25.039440 16945 heartbeater.cc:511] Master 127.2.30.190:36255 requested a full tablet report, sending...
I20260808 10:10:25.040115 16470 ts_manager.cc:194] Registered new tserver with Master: e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833)
I20260808 10:10:25.040635 16470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:50681
I20260808 10:10:25.044553 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:10:25.053843 16512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.046200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46870 (local address 127.2.30.190:36255)
0808 10:10:25.046271 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.046272 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.046419 (+ 147us) server_negotiation.cc:407] Connection header received
0808 10:10:25.046443 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.046443 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.046454 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.046497 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:25.047207 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.047721 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.048603 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.048807 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.048873 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.050176 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:25.050237 (+ 61us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:25.050238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:25.050245 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:25.052059 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:25.052560 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:25.052561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:25.052561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:25.052629 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:25.052893 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:25.052894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:25.052894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:25.053318 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:25.053431 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.053613 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.053699 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:10:25.056185 16470 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:10:25.057984 16470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:10:25.066419 16958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.061228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:33175 (local address 127.0.0.1:56380)
0808 10:10:25.061557 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:25.061616 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:25.061700 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:25.062120 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:25.062123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:25.062126 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:25.062226 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.062229 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.063650 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.063651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.064865 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.064867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.066003 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.066004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.066107 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.066125 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 10:10:25.066247 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 10:10:25.066315 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":92,"threads_started":1}
I20260808 10:10:25.066766 16960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.061522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56380 (local address 127.2.30.129:33175)
0808 10:10:25.061916 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.061921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.061934 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:10:25.061983 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.061986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.062006 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.062105 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:25.062548 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.063547 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.064972 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.065910 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.065982 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.066050 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.066637 (+ 587us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.066689 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":81,"threads_started":1}
I20260808 10:10:25.066727 16956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.061312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:35479 (local address 127.0.0.1:48994)
0808 10:10:25.061771 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:25.061780 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:25.061822 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:25.063130 (+ 1308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:25.063131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:25.063133 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:25.063196 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.063198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.064589 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.064591 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.065604 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.065608 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.066505 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.066506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.066581 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.066587 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 10:10:25.066621 (+ 34us) client_negotiation.cc:240] Negotiation successful
0808 10:10:25.066666 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":80,"threads_started":1}
I20260808 10:10:25.067235 16961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.061553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48994 (local address 127.2.30.130:35479)
0808 10:10:25.061759 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.062173 (+ 414us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.062186 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:10:25.062328 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.062333 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.062355 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.063172 (+ 817us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:25.063272 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.064465 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.065706 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.066403 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.066818 (+ 415us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.066895 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.067107 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.067158 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":81,"threads_started":1}
I20260808 10:10:25.067404 16957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.061196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:37833 (local address 127.0.0.1:44340)
0808 10:10:25.062912 (+ 1716us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:25.062924 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:25.063032 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:25.063318 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:25.063319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:25.063322 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:25.063370 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.063372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.064628 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.064629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.065696 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.065700 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.066780 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.066781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.066852 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.066857 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:10:25.066912 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 10:10:25.067330 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":82,"threads_started":1}
I20260808 10:10:25.067730 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.061306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44340 (local address 127.2.30.131:37833)
0808 10:10:25.061944 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.061948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.063040 (+ 1092us) server_negotiation.cc:407] Connection header received
0808 10:10:25.063117 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.063120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.063138 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.063215 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:25.063452 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.064543 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.065836 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.066621 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.066686 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.066742 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.067579 (+ 837us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.067634 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":103,"threads_started":1}
I20260808 10:10:25.067790 16599 tablet_service.cc:1467] Processing CreateTablet for tablet c26518611f814ee68fed0b2620c76810 (DEFAULT_TABLE table=test-table [id=21557b7334214dc39a2e492fcd8ce935]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:10:25.067910 16739 tablet_service.cc:1467] Processing CreateTablet for tablet c26518611f814ee68fed0b2620c76810 (DEFAULT_TABLE table=test-table [id=21557b7334214dc39a2e492fcd8ce935]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:10:25.068264 16599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c26518611f814ee68fed0b2620c76810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:25.068305 16739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c26518611f814ee68fed0b2620c76810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:25.068447 16877 tablet_service.cc:1467] Processing CreateTablet for tablet c26518611f814ee68fed0b2620c76810 (DEFAULT_TABLE table=test-table [id=21557b7334214dc39a2e492fcd8ce935]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:10:25.068769 16877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c26518611f814ee68fed0b2620c76810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:25.072083 16962 tablet_bootstrap.cc:492] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137: Bootstrap starting.
I20260808 10:10:25.072451 16964 tablet_bootstrap.cc:492] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b: Bootstrap starting.
I20260808 10:10:25.073069 16962 tablet_bootstrap.cc:654] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:25.073206 16964 tablet_bootstrap.cc:654] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:25.073626 16962 log.cc:824] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:25.073699 16964 log.cc:824] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:25.074120 16963 tablet_bootstrap.cc:492] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: Bootstrap starting.
I20260808 10:10:25.074570 16964 tablet_bootstrap.cc:492] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b: No bootstrap required, opened a new log
I20260808 10:10:25.074579 16962 tablet_bootstrap.cc:492] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137: No bootstrap required, opened a new log
I20260808 10:10:25.074647 16964 ts_tablet_manager.cc:1397] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:10:25.074649 16962 ts_tablet_manager.cc:1397] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:10:25.074860 16963 tablet_bootstrap.cc:654] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:25.075474 16963 log.cc:824] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:25.076418 16963 tablet_bootstrap.cc:492] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: No bootstrap required, opened a new log
I20260808 10:10:25.076501 16963 ts_tablet_manager.cc:1397] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 10:10:25.076653 16962 raft_consensus.cc:348] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.076838 16962 raft_consensus.cc:374] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:25.076881 16962 raft_consensus.cc:729] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8b8c6016b1648cab00c272f178a0137, State: Initialized, Role: FOLLOWER
I20260808 10:10:25.077035 16962 consensus_queue.cc:260] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.076906 16964 raft_consensus.cc:348] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.077188 16964 raft_consensus.cc:374] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:25.077224 16964 raft_consensus.cc:729] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e586a293ae614b3f8a1b77faf45c331b, State: Initialized, Role: FOLLOWER
I20260808 10:10:25.077328 16964 consensus_queue.cc:260] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.077389 16962 ts_tablet_manager.cc:1428] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:10:25.077522 16807 heartbeater.cc:503] Master 127.2.30.190:36255 was elected leader, sending a full tablet report...
I20260808 10:10:25.077596 16964 ts_tablet_manager.cc:1428] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:10:25.077726 16945 heartbeater.cc:503] Master 127.2.30.190:36255 was elected leader, sending a full tablet report...
I20260808 10:10:25.079164 16963 raft_consensus.cc:348] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.079367 16963 raft_consensus.cc:374] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:25.079416 16963 raft_consensus.cc:729] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf56168db55404981e1183fb713760c, State: Initialized, Role: FOLLOWER
I20260808 10:10:25.079546 16963 consensus_queue.cc:260] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.079845 16963 ts_tablet_manager.cc:1428] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260808 10:10:25.079888 16669 heartbeater.cc:503] Master 127.2.30.190:36255 was elected leader, sending a full tablet report...
I20260808 10:10:25.118793 16969 raft_consensus.cc:482] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:10:25.118953 16969 raft_consensus.cc:504] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.119460 16969 leader_election.cc:288] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833), bdf56168db55404981e1183fb713760c (127.2.30.129:33175)
I20260808 10:10:25.124223 16960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.119868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:52773 (local address 127.2.30.129:33175)
0808 10:10:25.119984 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.119987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.120151 (+ 164us) server_negotiation.cc:407] Connection header received
0808 10:10:25.120206 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.120208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.120221 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.120299 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:25.120468 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.121166 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.123090 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.123674 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.123786 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.123800 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.124086 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.124145 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 10:10:25.124223 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.119830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:54223 (local address 127.2.30.131:37833)
0808 10:10:25.119947 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.119950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.119959 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:10:25.119992 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.119993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.120003 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.120057 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:25.120369 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.121156 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.123081 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.123744 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.123818 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.123826 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.124086 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.124145 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 10:10:25.124230 16809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.119722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:37833 (local address 127.2.30.130:54223)
0808 10:10:25.119800 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:25.119815 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:25.119873 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:25.120103 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:25.120104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:25.120107 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:25.120194 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.120197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.121510 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.121512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.122915 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.122920 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.123855 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.123857 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.123957 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.123965 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:10:25.124037 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 10:10:25.124097 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":58}
I20260808 10:10:25.124243 16972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.119725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:33175 (local address 127.2.30.130:52773)
0808 10:10:25.120077 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:25.120087 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:25.120144 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:25.120331 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:25.120332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:25.120334 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:25.120388 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.120390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.121510 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.121512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.122915 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.122920 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.123859 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.123860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.123957 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.123966 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 10:10:25.124037 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 10:10:25.124097 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":255,"thread_start_us":149,"threads_started":1}
I20260808 10:10:25.124770 16896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c26518611f814ee68fed0b2620c76810" candidate_uuid: "c8b8c6016b1648cab00c272f178a0137" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e586a293ae614b3f8a1b77faf45c331b" is_pre_election: true
I20260808 10:10:25.124773 16622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c26518611f814ee68fed0b2620c76810" candidate_uuid: "c8b8c6016b1648cab00c272f178a0137" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf56168db55404981e1183fb713760c" is_pre_election: true
I20260808 10:10:25.124974 16896 raft_consensus.cc:2454] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8b8c6016b1648cab00c272f178a0137 in term 0.
I20260808 10:10:25.124985 16622 raft_consensus.cc:2454] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8b8c6016b1648cab00c272f178a0137 in term 0.
I20260808 10:10:25.125252 16692 leader_election.cc:302] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8b8c6016b1648cab00c272f178a0137, e586a293ae614b3f8a1b77faf45c331b; no voters:
I20260808 10:10:25.125497 16969 raft_consensus.cc:2781] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:10:25.125567 16969 raft_consensus.cc:482] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:10:25.125597 16969 raft_consensus.cc:3037] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:25.126452 16969 raft_consensus.cc:504] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.126658 16969 leader_election.cc:288] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [CANDIDATE]: Term 1 election: Requested vote from peers e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833), bdf56168db55404981e1183fb713760c (127.2.30.129:33175)
I20260808 10:10:25.126852 16896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c26518611f814ee68fed0b2620c76810" candidate_uuid: "c8b8c6016b1648cab00c272f178a0137" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e586a293ae614b3f8a1b77faf45c331b"
I20260808 10:10:25.126873 16622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c26518611f814ee68fed0b2620c76810" candidate_uuid: "c8b8c6016b1648cab00c272f178a0137" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf56168db55404981e1183fb713760c"
I20260808 10:10:25.126967 16896 raft_consensus.cc:3037] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:25.127030 16622 raft_consensus.cc:3037] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:25.127807 16896 raft_consensus.cc:2454] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8b8c6016b1648cab00c272f178a0137 in term 1.
I20260808 10:10:25.127846 16622 raft_consensus.cc:2454] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8b8c6016b1648cab00c272f178a0137 in term 1.
I20260808 10:10:25.128053 16692 leader_election.cc:302] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8b8c6016b1648cab00c272f178a0137, e586a293ae614b3f8a1b77faf45c331b; no voters:
I20260808 10:10:25.128173 16969 raft_consensus.cc:2781] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:10:25.128401 16969 raft_consensus.cc:686] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [term 1 LEADER]: Becoming Leader. State: Replica: c8b8c6016b1648cab00c272f178a0137, State: Running, Role: LEADER
I20260808 10:10:25.128526 16969 consensus_queue.cc:237] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.129734 16469 catalog_manager.cc:5495] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8b8c6016b1648cab00c272f178a0137 (127.2.30.130). New cstate: current_term: 1 leader_uuid: "c8b8c6016b1648cab00c272f178a0137" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } health_report { overall_health: UNKNOWN } } }
I20260808 10:10:25.140144 16961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.137622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49010 (local address 127.2.30.130:35479)
0808 10:10:25.137680 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.137682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.137692 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:10:25.137764 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.137765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.137788 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.137847 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:10:25.137982 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.138504 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.139404 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.139600 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.139658 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.139810 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:10:25.139931 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:10:25.139997 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.140038 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.140058 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260808 10:10:25.140774 16670 tablet.cc:2369] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:10:25.142447 16622 raft_consensus.cc:1261] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Refusing update from remote peer c8b8c6016b1648cab00c272f178a0137: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:10:25.142447 16896 raft_consensus.cc:1261] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b [term 1 FOLLOWER]: Refusing update from remote peer c8b8c6016b1648cab00c272f178a0137: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:10:25.142885 16969 consensus_queue.cc:1035] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:10:25.143102 16973 consensus_queue.cc:1035] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:10:25.146145 16976 mvcc.cc:205] Tried to move back new op lower bound from 7316208947780186112 to 7316208947727331328. Current Snapshot: MvccSnapshot[applied={T|T < 7316208947780186112}]
I20260808 10:10:25.146839 16978 mvcc.cc:205] Tried to move back new op lower bound from 7316208947780186112 to 7316208947727331328. Current Snapshot: MvccSnapshot[applied={T|T < 7316208947780186112}]
I20260808 10:10:25.147203 16977 mvcc.cc:205] Tried to move back new op lower bound from 7316208947780186112 to 7316208947727331328. Current Snapshot: MvccSnapshot[applied={T|T < 7316208947780186112 or (T in {7316208947780186112})}]
I20260808 10:10:25.149734 16469 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:10:25.149849 16469 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:10:25.150728 16469 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102: Sending DeleteTablet for 3 replicas of tablet c26518611f814ee68fed0b2620c76810
I20260808 10:10:25.151109 16877 tablet_service.cc:1514] Processing DeleteTablet for tablet c26518611f814ee68fed0b2620c76810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:10:25 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:44340
I20260808 10:10:25.151217 16599 tablet_service.cc:1514] Processing DeleteTablet for tablet c26518611f814ee68fed0b2620c76810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:10:25 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:56380
I20260808 10:10:25.151198 16739 tablet_service.cc:1514] Processing DeleteTablet for tablet c26518611f814ee68fed0b2620c76810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:10:25 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:48994
I20260808 10:10:25.151367 16987 tablet_replica.cc:330] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b: stopping tablet replica
I20260808 10:10:25.151460 16988 tablet_replica.cc:330] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: stopping tablet replica
I20260808 10:10:25.151477 16987 raft_consensus.cc:2229] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:10:25.151571 16988 raft_consensus.cc:2229] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:10:25.151717 16988 raft_consensus.cc:2258] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:10:25.151724 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 16536
I20260808 10:10:25.151772 16987 raft_consensus.cc:2258] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:10:25.153882 16989 tablet_replica.cc:330] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137: stopping tablet replica
I20260808 10:10:25.154129 16989 raft_consensus.cc:2229] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:10:25.154348 16989 raft_consensus.cc:2258] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:10:25.155300 16987 ts_tablet_manager.cc:1905] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:10:25.155387 16989 ts_tablet_manager.cc:1905] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:10:25.157198 16987 ts_tablet_manager.cc:1918] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:10:25.157299 16987 log.cc:1196] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/c26518611f814ee68fed0b2620c76810
I20260808 10:10:25.157618 16987 ts_tablet_manager.cc:1939] T c26518611f814ee68fed0b2620c76810 P e586a293ae614b3f8a1b77faf45c331b: Deleting consensus metadata
I20260808 10:10:25.157765 16989 ts_tablet_manager.cc:1918] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:10:25.157851 16989 log.cc:1196] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/c26518611f814ee68fed0b2620c76810
I20260808 10:10:25.158171 16989 ts_tablet_manager.cc:1939] T c26518611f814ee68fed0b2620c76810 P c8b8c6016b1648cab00c272f178a0137: Deleting consensus metadata
I20260808 10:10:25.158425 16454 catalog_manager.cc:4841] TS e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833): tablet c26518611f814ee68fed0b2620c76810 (table test-table [id=21557b7334214dc39a2e492fcd8ce935]) successfully deleted
I20260808 10:10:25.159541 16455 catalog_manager.cc:4841] TS c8b8c6016b1648cab00c272f178a0137 (127.2.30.130:35479): tablet c26518611f814ee68fed0b2620c76810 (table test-table [id=21557b7334214dc39a2e492fcd8ce935]) successfully deleted
W20260808 10:10:25.164913 16457 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:33175 (error 108)
I20260808 10:10:25.166147 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 16675
I20260808 10:10:25.166353 16957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.166179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:33175 (local address 127.0.0.1:56384)
0808 10:10:25.166257 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:25.166309 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:33175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 10:10:25.167596 16457 catalog_manager.cc:4584] TS bdf56168db55404981e1183fb713760c (127.2.30.129:33175): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c26518611f814ee68fed0b2620c76810: Network error: Client connection negotiation failed: client connection to 127.2.30.129:33175: connect: Connection refused (error 111)
I20260808 10:10:25.173511 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 16812
I20260808 10:10:25.179845 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 16438
I20260808 10:10:25.187785 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:36255
--webserver_interface=127.2.30.190
--webserver_port=44065
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:36255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:25.278869 16992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:25.279147 16992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:25.279173 16992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:25.280587 16992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:10:25.280661 16992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:25.280676 16992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:25.280689 16992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:10:25.280702 16992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:10:25.282575 16992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:36255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:36255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=44065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:25.282913 16992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:25.283226 16992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:25.295359 16998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:25.295584 16999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:25.295359 17001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:25.295707 16992 server_base.cc:1034] running on GCE node
I20260808 10:10:25.296068 16992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:25.296448 16992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:25.297665 16992 hybrid_clock.cc:648] HybridClock initialized: now 1786183825297644 us; error 58 us; skew 500 ppm
Aug 08 10:10:25 dist-test-slave-w206 krb5kdc[16425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183825, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:25.301077 16992 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:10:25.301626 16992 webserver.cc:460] Webserver started at http://127.2.30.190:44065/ using document root <none> and password file <none>
I20260808 10:10:25.301921 16992 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:25.302023 16992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:25.303738 16992 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 10:10:25.304641 17008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:25.304904 16992 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:10:25.305027 16992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "27a09832204248caac5ee1bc22caa102"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "31b21137f1272ab7ccf0aabc257a3dbb"
server_key_iv: "b2e3062134e1c25ef0a257dba199d82c"
server_key_version: "encryptionkey@0"
I20260808 10:10:25.305495 16992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:25.315850 16992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:25.318625 16992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:25.318809 16992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:25.323694 16992 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:36255
I20260808 10:10:25.323762 17060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:36255 every 8 connection(s)
I20260808 10:10:25.324245 16992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:10:25.324728 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 16992
I20260808 10:10:25.325253 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:33175
--local_ip_for_outbound_sockets=127.2.30.129
--tserver_master_addrs=127.2.30.190:36255
--webserver_port=45453
--webserver_interface=127.2.30.129
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:10:25.327262 17061 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:10:25.328075 17061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102: Bootstrap starting.
I20260808 10:10:25.338460 17061 log.cc:824] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:25.341056 17061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:10:25.341456 17061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102: Bootstrap complete.
I20260808 10:10:25.344365 17061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } }
I20260808 10:10:25.344645 17061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a09832204248caac5ee1bc22caa102, State: Initialized, Role: FOLLOWER
I20260808 10:10:25.344897 17061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } }
I20260808 10:10:25.345311 17061 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:10:25.345566 17061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:10:25.345687 17061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:10:25.347106 17061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } }
I20260808 10:10:25.347216 17061 leader_election.cc:302] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27a09832204248caac5ee1bc22caa102; no voters:
I20260808 10:10:25.347470 17061 leader_election.cc:288] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:10:25.347517 17065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:10:25.347651 17065 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 2 LEADER]: Becoming Leader. State: Replica: 27a09832204248caac5ee1bc22caa102, State: Running, Role: LEADER
I20260808 10:10:25.347760 17065 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } }
I20260808 10:10:25.347795 17061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:10:25.348104 17065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27a09832204248caac5ee1bc22caa102. Latest consensus state: current_term: 2 leader_uuid: "27a09832204248caac5ee1bc22caa102" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } } }
I20260808 10:10:25.348109 17067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "27a09832204248caac5ee1bc22caa102" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } } }
I20260808 10:10:25.348193 17065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [sys.catalog]: This master's current role is: LEADER
I20260808 10:10:25.348253 17067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [sys.catalog]: This master's current role is: LEADER
I20260808 10:10:25.348527 17070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:10:25.349488 17070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:10:25.349637 17070 catalog_manager.cc:1259] Loaded cluster ID: fbbbfb597ed84ddda1734996d00f91d6
I20260808 10:10:25.349678 17070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:10:25.350025 17070 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 10:10:25.353164 17070 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 10:10:25.355450 17070 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:10:25.355715 17070 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102: Loaded TSK: 0
I20260808 10:10:25.355947 17070 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:10:25.432852 17063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:25.433080 17063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:25.433109 17063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:25.434448 17063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:25.434508 17063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:25.434540 17063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:10:25.436285 17063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:33175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=45453
--tserver_master_addrs=127.2.30.190:36255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:25.436568 17063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:25.436918 17063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:25.443799 17090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:25.443857 17063 server_base.cc:1034] running on GCE node
W20260808 10:10:25.443820 17089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:25.444124 17092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:25.444459 17063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:25.444887 17063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:25.446043 17063 hybrid_clock.cc:648] HybridClock initialized: now 1786183825446018 us; error 44 us; skew 500 ppm
Aug 08 10:10:25 dist-test-slave-w206 krb5kdc[16425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183825, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:25.449411 17063 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:10:25.450037 17063 webserver.cc:460] Webserver started at http://127.2.30.129:45453/ using document root <none> and password file <none>
I20260808 10:10:25.450325 17063 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:25.450433 17063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:25.452152 17063 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:25.452996 17099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:25.453215 17063 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:25.453341 17063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "bdf56168db55404981e1183fb713760c"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "4ad3568d7c8112d3df672f1ae24735c9"
server_key_iv: "1c3918e6fdaff38a09d6b4d6c9f04355"
server_key_version: "encryptionkey@0"
I20260808 10:10:25.453738 17063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:25.467697 17063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:25.470458 17063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:25.470671 17063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:25.471052 17063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:25.471676 17106 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:10:25.473769 17063 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:10:25.473848 17063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 10:10:25.473879 17063 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:10:25.474789 17063 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:10:25.474839 17063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 10:10:25.474891 17106 tablet_bootstrap.cc:492] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: Bootstrap starting.
I20260808 10:10:25.483655 17063 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:33175
I20260808 10:10:25.483881 17216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:33175 every 8 connection(s)
I20260808 10:10:25.484208 17063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:10:25.488618 17106 log.cc:824] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:25.491397 17106 tablet_bootstrap.cc:492] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:10:25.491750 17106 tablet_bootstrap.cc:492] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: Bootstrap complete.
I20260808 10:10:25.493016 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 17063
I20260808 10:10:25.493237 17106 ts_tablet_manager.cc:1397] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.012s
I20260808 10:10:25.493335 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:35479
--local_ip_for_outbound_sockets=127.2.30.130
--tserver_master_addrs=127.2.30.190:36255
--webserver_port=44741
--webserver_interface=127.2.30.130
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:10:25.494215 17106 raft_consensus.cc:348] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.494438 17106 raft_consensus.cc:729] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf56168db55404981e1183fb713760c, State: Initialized, Role: FOLLOWER
I20260808 10:10:25.494582 17106 consensus_queue.cc:260] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.494904 17106 ts_tablet_manager.cc:1428] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
Aug 08 10:10:25 dist-test-slave-w206 krb5kdc[16425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183825, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:25.500703 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.485199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:56609 (local address 127.2.30.190:36255)
0808 10:10:25.485433 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.485443 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.485483 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:10:25.486371 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.486379 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.486441 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.486601 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:25.487720 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.489080 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.490622 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.491306 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.491444 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.496232 (+ 4788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:25.496243 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:25.496243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:25.496256 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:25.498408 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:25.499048 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:25.499052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:25.499053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:25.499180 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:25.499520 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:25.499521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:25.499521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:25.499817 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:25.499919 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.500338 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.500488 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":100,"threads_started":1}
I20260808 10:10:25.500771 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.485067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:56609)
0808 10:10:25.485279 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:25.485326 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:25.486098 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:25.486687 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:25.486693 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:25.487229 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:25.487544 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.487554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.489250 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.489252 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.490508 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.490511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.491444 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.491445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.491614 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.492289 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:25.492301 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:25.496064 (+ 3763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:25.498518 (+ 2454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:25.498528 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:25.498531 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:25.498911 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:25.499258 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:25.499259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:25.499260 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:25.499431 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:25.499910 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:25.499918 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:25.500113 (+ 195us) client_negotiation.cc:769] Sending connection context
0808 10:10:25.500262 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 10:10:25.500508 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":126,"threads_started":1}
I20260808 10:10:25.501943 17217 heartbeater.cc:347] Connected to a master server at 127.2.30.190:36255
I20260808 10:10:25.502056 17217 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:25.502254 17217 heartbeater.cc:511] Master 127.2.30.190:36255 requested a full tablet report, sending...
I20260808 10:10:25.503273 17025 ts_manager.cc:194] Registered new tserver with Master: bdf56168db55404981e1183fb713760c (127.2.30.129:33175)
I20260808 10:10:25.504683 17025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:56609
I20260808 10:10:25.505945 17217 heartbeater.cc:503] Master 127.2.30.190:36255 was elected leader, sending a full tablet report...
I20260808 10:10:25.511441 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.504075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:33175 (local address 127.0.0.1:56398)
0808 10:10:25.505333 (+ 1258us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:25.505379 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:25.505465 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:25.507328 (+ 1863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:25.507332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:25.507335 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:25.507411 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.507413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.508795 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.508796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.510150 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.510154 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.511107 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.511108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.511198 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.511212 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 10:10:25.511288 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:10:25.511343 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1218,"thread_start_us":110,"threads_started":1}
I20260808 10:10:25.511445 17225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.505101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56398 (local address 127.2.30.129:33175)
0808 10:10:25.507118 (+ 2017us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.507124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.507137 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:10:25.507180 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.507183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.507200 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.507305 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:25.507504 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.508694 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.510240 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.511024 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.511103 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.511164 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.511314 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.511359 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1974,"thread_start_us":108,"threads_started":1}
I20260808 10:10:25.512320 17150 tablet_service.cc:1514] Processing DeleteTablet for tablet c26518611f814ee68fed0b2620c76810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:10:25 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:56398
I20260808 10:10:25.515166 17226 tablet_replica.cc:330] stopping tablet replica
I20260808 10:10:25.515311 17226 raft_consensus.cc:2229] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:10:25.515390 17226 raft_consensus.cc:2258] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:10:25.515769 17226 ts_tablet_manager.cc:1905] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:10:25.518611 17226 ts_tablet_manager.cc:1918] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:10:25.518723 17226 log.cc:1196] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal/wals/c26518611f814ee68fed0b2620c76810
I20260808 10:10:25.519007 17226 ts_tablet_manager.cc:1939] T c26518611f814ee68fed0b2620c76810 P bdf56168db55404981e1183fb713760c: Deleting consensus metadata
I20260808 10:10:25.519701 17012 catalog_manager.cc:4841] TS bdf56168db55404981e1183fb713760c (127.2.30.129:33175): tablet c26518611f814ee68fed0b2620c76810 (table test-table [id=21557b7334214dc39a2e492fcd8ce935]) successfully deleted
W20260808 10:10:25.588754 17223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:25.589080 17223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:25.589123 17223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:25.590620 17223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:25.590682 17223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:25.590713 17223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:10:25.592446 17223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:35479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=44741
--tserver_master_addrs=127.2.30.190:36255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:25.592758 17223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:25.593040 17223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:25.599144 17232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:25.599110 17231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:25.599375 17223 server_base.cc:1034] running on GCE node
W20260808 10:10:25.599119 17234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:25.599733 17223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:25.600132 17223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:25.601353 17223 hybrid_clock.cc:648] HybridClock initialized: now 1786183825601329 us; error 43 us; skew 500 ppm
Aug 08 10:10:25 dist-test-slave-w206 krb5kdc[16425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183825, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:25.604460 17223 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:10:25.604965 17223 webserver.cc:460] Webserver started at http://127.2.30.130:44741/ using document root <none> and password file <none>
I20260808 10:10:25.605247 17223 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:25.605317 17223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:25.606899 17223 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:10:25.607867 17241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:25.608052 17223 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:25.608167 17223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "c8b8c6016b1648cab00c272f178a0137"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "3a7035de113beda7dde2c15de00f50cf"
server_key_iv: "2e1b074a7cdb6b59915a01e0fb6ad080"
server_key_version: "encryptionkey@0"
I20260808 10:10:25.608606 17223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:25.627415 17223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:25.630113 17223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:25.630316 17223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:25.630625 17223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:25.631094 17223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:10:25.631152 17223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:25.631206 17223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:10:25.631240 17223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:25.637665 17223 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:35479
I20260808 10:10:25.637763 17358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:35479 every 8 connection(s)
I20260808 10:10:25.638175 17223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:10:25.641005 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 17223
I20260808 10:10:25.641399 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:37833
--local_ip_for_outbound_sockets=127.2.30.131
--tserver_master_addrs=127.2.30.190:36255
--webserver_port=34205
--webserver_interface=127.2.30.131
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:10:25 dist-test-slave-w206 krb5kdc[16425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183825, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:25.650552 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.639124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:42671 (local address 127.2.30.190:36255)
0808 10:10:25.639186 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.639188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.639417 (+ 229us) server_negotiation.cc:407] Connection header received
0808 10:10:25.639984 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.639985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.640001 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.640096 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:25.641239 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.641874 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.643011 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.643375 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.643457 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.646620 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:25.646622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:25.646622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:25.646630 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:25.648629 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:25.649384 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:25.649386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:25.649386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:25.649461 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:25.649758 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:25.649759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:25.649760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:25.649907 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:25.649992 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.650339 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.650448 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 10:10:25.650797 17361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.639105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:42671)
0808 10:10:25.639267 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:25.639317 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:25.639854 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:25.640157 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:25.640160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:25.640822 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:25.641049 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.641055 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.641994 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.641995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.642873 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.642876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.643496 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.643498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.643631 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.644510 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:25.644519 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:25.646484 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:25.648817 (+ 2333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:25.648846 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:25.648849 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:25.649294 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:25.649539 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:25.649540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:25.649540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:25.649684 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:25.649981 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:25.649990 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:25.650160 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 10:10:25.650314 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 10:10:25.650510 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260808 10:10:25.651432 17359 heartbeater.cc:347] Connected to a master server at 127.2.30.190:36255
I20260808 10:10:25.651573 17359 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:25.651826 17359 heartbeater.cc:511] Master 127.2.30.190:36255 requested a full tablet report, sending...
I20260808 10:10:25.652397 17025 ts_manager.cc:194] Registered new tserver with Master: c8b8c6016b1648cab00c272f178a0137 (127.2.30.130:35479)
I20260808 10:10:25.652845 17025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:42671
W20260808 10:10:25.732339 17362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:25.732570 17362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:25.732599 17362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:25.733855 17362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:25.733906 17362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:25.733933 17362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:10:25.735556 17362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:37833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=34205
--tserver_master_addrs=127.2.30.190:36255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:25.735777 17362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:25.735982 17362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:10:25.741927 17362 server_base.cc:1034] running on GCE node
W20260808 10:10:25.741892 17369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:25.741892 17367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:25.741915 17371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:25.742462 17362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:25.742795 17362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:25.744024 17362 hybrid_clock.cc:648] HybridClock initialized: now 1786183825744016 us; error 59 us; skew 500 ppm
Aug 08 10:10:25 dist-test-slave-w206 krb5kdc[16425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183825, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:25.746959 17362 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:10:25.747623 17362 webserver.cc:460] Webserver started at http://127.2.30.131:34205/ using document root <none> and password file <none>
I20260808 10:10:25.747956 17362 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:25.748062 17362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:25.749737 17362 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 10:10:25.750617 17379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:25.750887 17362 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:10:25.751053 17362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "e586a293ae614b3f8a1b77faf45c331b"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "1f3377ed638db8ed602b1fb38b643ed9"
server_key_iv: "773ff875570ce051703126f6272d074f"
server_key_version: "encryptionkey@0"
I20260808 10:10:25.751510 17362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:25.773586 17362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:25.776518 17362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:25.776660 17362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:25.776881 17362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:25.777340 17362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:10:25.777379 17362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:25.777403 17362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:10:25.777416 17362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:25.784219 17362 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:37833
I20260808 10:10:25.784471 17496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:37833 every 8 connection(s)
I20260808 10:10:25.784758 17362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:10:25.789009 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 17362
Aug 08 10:10:25 dist-test-slave-w206 krb5kdc[16425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183825, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:25.798233 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.785708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:42411 (local address 127.2.30.190:36255)
0808 10:10:25.785780 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.785782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.786001 (+ 219us) server_negotiation.cc:407] Connection header received
0808 10:10:25.786555 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.786556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.786568 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.786638 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:25.787736 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.788269 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.789782 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.790012 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.790109 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.793941 (+ 3832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:25.793943 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:25.793943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:25.793952 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:25.795953 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:25.796647 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:25.796648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:25.796649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:25.796724 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:25.797156 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:25.797157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:25.797158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:25.797481 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:25.797558 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.798058 (+ 500us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.798156 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:10:25.798581 17500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.785684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:42411)
0808 10:10:25.785851 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:25.785898 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:25.786430 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:25.786697 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:25.786701 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:25.787354 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:25.787565 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.787569 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.788453 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.788454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.789660 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.789664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.790141 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.790147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.790280 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.791048 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:25.791060 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:25.793806 (+ 2746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:25.796152 (+ 2346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:25.796167 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:25.796170 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:25.796545 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:25.796860 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:25.796862 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:25.796862 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:25.797051 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:25.797580 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:25.797592 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:25.797851 (+ 259us) client_negotiation.cc:769] Sending connection context
0808 10:10:25.798011 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 10:10:25.798224 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260808 10:10:25.799182 17497 heartbeater.cc:347] Connected to a master server at 127.2.30.190:36255
I20260808 10:10:25.799319 17497 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:25.799660 17497 heartbeater.cc:511] Master 127.2.30.190:36255 requested a full tablet report, sending...
I20260808 10:10:25.800098 17502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.789661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44084 (local address 127.2.30.190:36255)
0808 10:10:25.789966 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.789968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.789974 (+ 6us) server_negotiation.cc:407] Connection header received
0808 10:10:25.789992 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.789993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.790002 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.790052 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:25.790871 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.791646 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.793638 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.793998 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.794056 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.795896 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:25.795898 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:25.795898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:25.795907 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:25.798385 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:25.799113 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:25.799114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:25.799114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:25.799171 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:25.799442 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:25.799443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:25.799444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:25.799571 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:25.799796 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.799837 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.799972 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":94,"threads_started":1}
I20260808 10:10:25.800328 17025 ts_manager.cc:194] Registered new tserver with Master: e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833)
I20260808 10:10:25.800813 17025 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:42411
I20260808 10:10:25.802757 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:10:25.807582 17502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.803846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44098 (local address 127.2.30.190:36255)
0808 10:10:25.804021 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.804023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.804193 (+ 170us) server_negotiation.cc:407] Connection header received
0808 10:10:25.804345 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.804346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.804404 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.804646 (+ 242us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:10:25.804852 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.805431 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.806468 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.806711 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.806798 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.807046 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:10:25.807181 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:10:25.807295 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.807313 (+ 18us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.807349 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260808 10:10:25.808758 17024 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:10:25.814502 17024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:10:25.819271 17150 tablet_service.cc:1467] Processing CreateTablet for tablet a7b44a90fb4b4eefb0deb99d9fbb717e (DEFAULT_TABLE table=test-table [id=c73c1f42cc8e49e59c7ebbb408750a7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:10:25.819578 17150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7b44a90fb4b4eefb0deb99d9fbb717e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:25.821559 17106 tablet_bootstrap.cc:492] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c: Bootstrap starting.
I20260808 10:10:25.822417 17106 tablet_bootstrap.cc:654] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:25.823734 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.818515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:37833 (local address 127.0.0.1:34872)
0808 10:10:25.818596 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:25.818611 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:25.818661 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:25.819204 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:25.819205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:25.819208 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:25.819273 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.819276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.821372 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.821374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.822499 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.822502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.823501 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.823502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.823574 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.823579 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:10:25.823629 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 10:10:25.823671 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 10:10:25.823747 17106 tablet_bootstrap.cc:492] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c: No bootstrap required, opened a new log
I20260808 10:10:25.823822 17106 ts_tablet_manager.cc:1397] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 10:10:25.824015 17106 raft_consensus.cc:348] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.824091 17106 raft_consensus.cc:374] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:25.824174 17106 raft_consensus.cc:729] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf56168db55404981e1183fb713760c, State: Initialized, Role: FOLLOWER
I20260808 10:10:25.824203 17504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.818643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34872 (local address 127.2.30.131:37833)
0808 10:10:25.818957 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.818964 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.818975 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:10:25.819050 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.819053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.819076 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.819365 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:25.819476 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.821233 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.822598 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.823399 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.823900 (+ 501us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.823978 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.824008 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.824074 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":166,"threads_started":1}
I20260808 10:10:25.824230 17106 consensus_queue.cc:260] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.824517 17106 ts_tablet_manager.cc:1428] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:25.824924 17505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.818924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:35479 (local address 127.0.0.1:34528)
0808 10:10:25.819130 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:25.819144 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:25.819208 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:25.819690 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:25.819691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:25.819695 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:25.819753 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.819755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.820955 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.820956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.822274 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.822278 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.823502 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.823504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.824680 (+ 1176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.824687 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:10:25.824773 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 10:10:25.824828 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":135,"threads_started":1}
I20260808 10:10:25.824957 17506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.819072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34528 (local address 127.2.30.130:35479)
0808 10:10:25.819327 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.819331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.819340 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:10:25.819466 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.819468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.819487 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.819575 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:25.819846 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.820829 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.822387 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.823365 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.823462 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.823537 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.824816 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.824872 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":93,"threads_started":1}
I20260808 10:10:25.825299 17430 tablet_service.cc:1467] Processing CreateTablet for tablet a7b44a90fb4b4eefb0deb99d9fbb717e (DEFAULT_TABLE table=test-table [id=c73c1f42cc8e49e59c7ebbb408750a7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:10:25.825714 17430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7b44a90fb4b4eefb0deb99d9fbb717e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:25.825894 17291 tablet_service.cc:1467] Processing CreateTablet for tablet a7b44a90fb4b4eefb0deb99d9fbb717e (DEFAULT_TABLE table=test-table [id=c73c1f42cc8e49e59c7ebbb408750a7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:10:25.826164 17291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7b44a90fb4b4eefb0deb99d9fbb717e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:25.828848 17508 tablet_bootstrap.cc:492] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b: Bootstrap starting.
I20260808 10:10:25.829660 17508 tablet_bootstrap.cc:654] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:25.830190 17509 tablet_bootstrap.cc:492] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137: Bootstrap starting.
I20260808 10:10:25.830495 17508 log.cc:824] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:25.830875 17509 tablet_bootstrap.cc:654] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:25.831360 17508 tablet_bootstrap.cc:492] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b: No bootstrap required, opened a new log
I20260808 10:10:25.831451 17509 log.cc:824] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:25.831555 17508 ts_tablet_manager.cc:1397] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:10:25.832432 17509 tablet_bootstrap.cc:492] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137: No bootstrap required, opened a new log
I20260808 10:10:25.832525 17509 ts_tablet_manager.cc:1397] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:10:25.833535 17508 raft_consensus.cc:348] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.833712 17508 raft_consensus.cc:374] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:25.833739 17508 raft_consensus.cc:729] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e586a293ae614b3f8a1b77faf45c331b, State: Initialized, Role: FOLLOWER
I20260808 10:10:25.833900 17508 consensus_queue.cc:260] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.834293 17508 ts_tablet_manager.cc:1428] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:10:25.834293 17497 heartbeater.cc:503] Master 127.2.30.190:36255 was elected leader, sending a full tablet report...
I20260808 10:10:25.834466 17509 raft_consensus.cc:348] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.834656 17509 raft_consensus.cc:374] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:25.834694 17509 raft_consensus.cc:729] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8b8c6016b1648cab00c272f178a0137, State: Initialized, Role: FOLLOWER
I20260808 10:10:25.834825 17509 consensus_queue.cc:260] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.835155 17509 ts_tablet_manager.cc:1428] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260808 10:10:25.835187 17359 heartbeater.cc:503] Master 127.2.30.190:36255 was elected leader, sending a full tablet report...
W20260808 10:10:25.888952 17360 tablet.cc:2369] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:10:25.891157 17222 raft_consensus.cc:482] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:10:25.891348 17222 raft_consensus.cc:504] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.891804 17222 leader_election.cc:288] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833), c8b8c6016b1648cab00c272f178a0137 (127.2.30.130:35479)
I20260808 10:10:25.895167 17506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.892073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:41285 (local address 127.2.30.130:35479)
0808 10:10:25.892119 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.892121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.892391 (+ 270us) server_negotiation.cc:407] Connection header received
0808 10:10:25.892458 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.892459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.892474 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.892549 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:25.892710 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.893206 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.894310 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.894738 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.894835 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.894848 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.895034 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.895084 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:10:25.895177 17504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.892054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:39933 (local address 127.2.30.131:37833)
0808 10:10:25.892119 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.892121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.892133 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:10:25.892195 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.892196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.892205 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.892268 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:25.892777 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.893252 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.894294 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.894729 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.894824 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.894834 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.895041 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.895082 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:10:25.895254 17514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.891989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:35479 (local address 127.2.30.129:41285)
0808 10:10:25.892310 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:25.892320 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:25.892403 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:25.892552 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:25.892553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:25.892555 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:25.892637 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.892639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.893314 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.893315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.894190 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.894193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.894855 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.894857 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.894929 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.894933 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:10:25.895025 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 10:10:25.895077 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":105,"thread_start_us":204,"threads_started":1}
I20260808 10:10:25.895202 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.891980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:37833 (local address 127.2.30.129:39933)
0808 10:10:25.892059 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:25.892072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:25.892126 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:25.892294 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:25.892296 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:25.892298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:25.892644 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.892647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.893343 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.893344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.894189 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:25.894192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.894824 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.894825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:25.894930 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.894934 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:10:25.895008 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 10:10:25.895077 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":72}
I20260808 10:10:25.895659 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7b44a90fb4b4eefb0deb99d9fbb717e" candidate_uuid: "bdf56168db55404981e1183fb713760c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8b8c6016b1648cab00c272f178a0137" is_pre_election: true
I20260808 10:10:25.895781 17450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7b44a90fb4b4eefb0deb99d9fbb717e" candidate_uuid: "bdf56168db55404981e1183fb713760c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e586a293ae614b3f8a1b77faf45c331b" is_pre_election: true
I20260808 10:10:25.895856 17311 raft_consensus.cc:2454] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf56168db55404981e1183fb713760c in term 0.
I20260808 10:10:25.895926 17450 raft_consensus.cc:2454] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdf56168db55404981e1183fb713760c in term 0.
I20260808 10:10:25.896158 17101 leader_election.cc:302] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf56168db55404981e1183fb713760c, c8b8c6016b1648cab00c272f178a0137; no voters:
I20260808 10:10:25.896363 17222 raft_consensus.cc:2781] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:10:25.896476 17222 raft_consensus.cc:482] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:10:25.896509 17222 raft_consensus.cc:3037] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:25.897395 17222 raft_consensus.cc:504] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.897593 17222 leader_election.cc:288] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 1 election: Requested vote from peers e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833), c8b8c6016b1648cab00c272f178a0137 (127.2.30.130:35479)
I20260808 10:10:25.897794 17311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7b44a90fb4b4eefb0deb99d9fbb717e" candidate_uuid: "bdf56168db55404981e1183fb713760c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8b8c6016b1648cab00c272f178a0137"
I20260808 10:10:25.897878 17450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7b44a90fb4b4eefb0deb99d9fbb717e" candidate_uuid: "bdf56168db55404981e1183fb713760c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e586a293ae614b3f8a1b77faf45c331b"
I20260808 10:10:25.897903 17311 raft_consensus.cc:3037] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:25.897998 17450 raft_consensus.cc:3037] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:25.898782 17450 raft_consensus.cc:2454] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf56168db55404981e1183fb713760c in term 1.
I20260808 10:10:25.898787 17311 raft_consensus.cc:2454] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdf56168db55404981e1183fb713760c in term 1.
I20260808 10:10:25.899049 17101 leader_election.cc:302] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf56168db55404981e1183fb713760c, c8b8c6016b1648cab00c272f178a0137; no voters:
I20260808 10:10:25.899155 17222 raft_consensus.cc:2781] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:10:25.899335 17222 raft_consensus.cc:686] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 LEADER]: Becoming Leader. State: Replica: bdf56168db55404981e1183fb713760c, State: Running, Role: LEADER
I20260808 10:10:25.899432 17222 consensus_queue.cc:237] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:25.900435 17025 catalog_manager.cc:5495] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c reported cstate change: term changed from 0 to 1, leader changed from <none> to bdf56168db55404981e1183fb713760c (127.2.30.129). New cstate: current_term: 1 leader_uuid: "bdf56168db55404981e1183fb713760c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } health_report { overall_health: HEALTHY } } }
I20260808 10:10:25.927618 17225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.924758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33548 (local address 127.2.30.129:33175)
0808 10:10:25.924865 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:25.924867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:25.924876 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:10:25.924921 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:25.924922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:25.924947 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:25.925026 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:10:25.925231 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.925724 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.926575 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:25.926766 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:25.926832 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:25.927081 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:10:25.927199 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:10:25.927260 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:25.927522 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 10:10:25.927548 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 10:10:25.929533 17311 raft_consensus.cc:1261] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137 [term 1 FOLLOWER]: Refusing update from remote peer bdf56168db55404981e1183fb713760c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:10:25.929569 17450 raft_consensus.cc:1261] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b [term 1 FOLLOWER]: Refusing update from remote peer bdf56168db55404981e1183fb713760c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:10:25.929901 17222 consensus_queue.cc:1035] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:10:25.930051 17515 consensus_queue.cc:1035] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:10:25.933911 17518 mvcc.cc:205] Tried to move back new op lower bound from 7316208951004381184 to 7316208950884806656. Current Snapshot: MvccSnapshot[applied={T|T < 7316208951004381184 or (T in {7316208951004381184})}]
I20260808 10:10:25.935657 17024 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:10:25.935779 17024 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:10:25.936631 17024 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102: Sending DeleteTablet for 3 replicas of tablet a7b44a90fb4b4eefb0deb99d9fbb717e
I20260808 10:10:25.936996 17150 tablet_service.cc:1514] Processing DeleteTablet for tablet a7b44a90fb4b4eefb0deb99d9fbb717e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:10:25 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:56398
I20260808 10:10:25.937011 17291 tablet_service.cc:1514] Processing DeleteTablet for tablet a7b44a90fb4b4eefb0deb99d9fbb717e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:10:25 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:34528
I20260808 10:10:25.937101 17226 tablet_replica.cc:330] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c: stopping tablet replica
I20260808 10:10:25.937083 17430 tablet_service.cc:1514] Processing DeleteTablet for tablet a7b44a90fb4b4eefb0deb99d9fbb717e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:10:25 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:34872
I20260808 10:10:25.937122 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 17063
I20260808 10:10:25.937151 17226 raft_consensus.cc:2229] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:10:25.937240 17226 raft_consensus.cc:2258] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:10:25.938179 17529 tablet_replica.cc:330] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b: stopping tablet replica
I20260808 10:10:25.938198 17528 tablet_replica.cc:330] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137: stopping tablet replica
I20260808 10:10:25.938298 17528 raft_consensus.cc:2229] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:10:25.938298 17529 raft_consensus.cc:2229] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:10:25.938409 17529 raft_consensus.cc:2258] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:10:25.938409 17528 raft_consensus.cc:2258] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:10:25.938930 17529 ts_tablet_manager.cc:1905] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:10:25.941504 17529 ts_tablet_manager.cc:1918] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:10:25.941641 17529 log.cc:1196] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/a7b44a90fb4b4eefb0deb99d9fbb717e
I20260808 10:10:25.941977 17529 ts_tablet_manager.cc:1939] T a7b44a90fb4b4eefb0deb99d9fbb717e P e586a293ae614b3f8a1b77faf45c331b: Deleting consensus metadata
I20260808 10:10:25.942618 17009 catalog_manager.cc:4841] TS e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833): tablet a7b44a90fb4b4eefb0deb99d9fbb717e (table test-table [id=c73c1f42cc8e49e59c7ebbb408750a7e]) successfully deleted
I20260808 10:10:25.944391 17528 ts_tablet_manager.cc:1905] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:10:25.946446 17528 ts_tablet_manager.cc:1918] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:10:25.946538 17528 log.cc:1196] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/a7b44a90fb4b4eefb0deb99d9fbb717e
I20260808 10:10:25.946847 17528 ts_tablet_manager.cc:1939] T a7b44a90fb4b4eefb0deb99d9fbb717e P c8b8c6016b1648cab00c272f178a0137: Deleting consensus metadata
I20260808 10:10:25.947569 17010 catalog_manager.cc:4841] TS c8b8c6016b1648cab00c272f178a0137 (127.2.30.130:35479): tablet a7b44a90fb4b4eefb0deb99d9fbb717e (table test-table [id=c73c1f42cc8e49e59c7ebbb408750a7e]) successfully deleted
W20260808 10:10:25.948380 17012 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:33175 (error 108)
I20260808 10:10:25.948899 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 17223
I20260808 10:10:25.949018 17505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:25.948877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:33175 (local address 127.0.0.1:33550)
0808 10:10:25.948919 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:25.948966 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:33175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260808 10:10:25.949136 17012 catalog_manager.cc:4584] TS bdf56168db55404981e1183fb713760c (127.2.30.129:33175): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a7b44a90fb4b4eefb0deb99d9fbb717e: Network error: Client connection negotiation failed: client connection to 127.2.30.129:33175: connect: Connection refused (error 111)
I20260808 10:10:25.956532 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 17362
I20260808 10:10:25.963495 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 16992
I20260808 10:10:25.971418 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:36255
--webserver_interface=127.2.30.190
--webserver_port=44065
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:36255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:26.065472 17531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:26.065752 17531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:26.065783 17531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:26.067309 17531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:10:26.067389 17531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:26.067406 17531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:26.067421 17531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:10:26.067436 17531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:10:26.069212 17531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:36255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:36255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=44065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:26.069484 17531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:26.069713 17531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:26.076798 17539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:26.076907 17531 server_base.cc:1034] running on GCE node
W20260808 10:10:26.076793 17537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:26.076789 17536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:26.077570 17531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:26.077953 17531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:26.079078 17531 hybrid_clock.cc:648] HybridClock initialized: now 1786183826079055 us; error 43 us; skew 500 ppm
Aug 08 10:10:26 dist-test-slave-w206 krb5kdc[16425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183826, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:26.082263 17531 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:10:26.082871 17531 webserver.cc:460] Webserver started at http://127.2.30.190:44065/ using document root <none> and password file <none>
I20260808 10:10:26.083158 17531 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:26.083210 17531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:26.084770 17531 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:26.085696 17546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:26.085997 17531 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:26.086084 17531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "27a09832204248caac5ee1bc22caa102"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "31b21137f1272ab7ccf0aabc257a3dbb"
server_key_iv: "b2e3062134e1c25ef0a257dba199d82c"
server_key_version: "encryptionkey@0"
I20260808 10:10:26.086516 17531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:26.096521 17531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:26.100049 17531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:26.100256 17531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:26.105875 17531 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:36255
I20260808 10:10:26.106206 17601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:36255 every 8 connection(s)
I20260808 10:10:26.106608 17531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:10:26.108147 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 17531
I20260808 10:10:26.108507 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:33175
--local_ip_for_outbound_sockets=127.2.30.129
--tserver_master_addrs=127.2.30.190:36255
--webserver_port=45453
--webserver_interface=127.2.30.129
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:10:26.108891 17602 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:10:26.109753 17602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102: Bootstrap starting.
I20260808 10:10:26.120458 17602 log.cc:824] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:26.123153 17602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:10:26.123528 17602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102: Bootstrap complete.
I20260808 10:10:26.126062 17602 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } }
I20260808 10:10:26.126338 17602 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a09832204248caac5ee1bc22caa102, State: Initialized, Role: FOLLOWER
I20260808 10:10:26.126571 17602 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } }
I20260808 10:10:26.126679 17602 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:10:26.126744 17602 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:10:26.126804 17602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 2 FOLLOWER]: Advancing to term 3
I20260808 10:10:26.128012 17602 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } }
I20260808 10:10:26.128149 17602 leader_election.cc:302] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27a09832204248caac5ee1bc22caa102; no voters:
I20260808 10:10:26.128381 17602 leader_election.cc:288] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 10:10:26.128475 17606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 10:10:26.128634 17606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [term 3 LEADER]: Becoming Leader. State: Replica: 27a09832204248caac5ee1bc22caa102, State: Running, Role: LEADER
I20260808 10:10:26.128721 17602 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:10:26.128729 17606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } }
I20260808 10:10:26.129084 17606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27a09832204248caac5ee1bc22caa102. Latest consensus state: current_term: 3 leader_uuid: "27a09832204248caac5ee1bc22caa102" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } } }
I20260808 10:10:26.129184 17606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [sys.catalog]: This master's current role is: LEADER
I20260808 10:10:26.129415 17607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "27a09832204248caac5ee1bc22caa102" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a09832204248caac5ee1bc22caa102" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36255 } } }
I20260808 10:10:26.129518 17607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27a09832204248caac5ee1bc22caa102 [sys.catalog]: This master's current role is: LEADER
I20260808 10:10:26.129845 17611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:10:26.130707 17611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:10:26.130821 17611 catalog_manager.cc:1259] Loaded cluster ID: fbbbfb597ed84ddda1734996d00f91d6
I20260808 10:10:26.130872 17611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 10:10:26.131510 17611 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fce0397a9ad google::LogMessage::Fail() at ??:0
@ 0x7fce0397d88a google::LogMessage::SendToLog() at ??:0
@ 0x7fce0397a4de google::LogMessage::Flush() at ??:0
@ 0x7fce0397ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fce0b8a40f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fce0b8ad8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fce058480d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fce0583d954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fce066a16db start_thread at ??:0
@ 0x7fce0247861f clone at ??:0
W20260808 10:10:26.216109 17604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:26.216404 17604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:26.216452 17604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:26.217880 17604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:26.217984 17604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:26.218056 17604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:10:26.220099 17604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:33175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=45453
--tserver_master_addrs=127.2.30.190:36255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:26.220460 17604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:26.220762 17604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:10:26.226804 17604 server_base.cc:1034] running on GCE node
W20260808 10:10:26.226742 17642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:26.226948 17640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:26.226739 17639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:26.227373 17604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:26.227710 17604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:26.228924 17604 hybrid_clock.cc:648] HybridClock initialized: now 1786183826228896 us; error 45 us; skew 500 ppm
Aug 08 10:10:26 dist-test-slave-w206 krb5kdc[16425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183826, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:26.232375 17604 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:10:26.233089 17604 webserver.cc:460] Webserver started at http://127.2.30.129:45453/ using document root <none> and password file <none>
I20260808 10:10:26.233438 17604 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:26.233525 17604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:26.236078 17604 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:10:26.237557 17649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:26.238672 17604 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 10:10:26.238788 17604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "bdf56168db55404981e1183fb713760c"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "4ad3568d7c8112d3df672f1ae24735c9"
server_key_iv: "1c3918e6fdaff38a09d6b4d6c9f04355"
server_key_version: "encryptionkey@0"
I20260808 10:10:26.239549 17604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:26.261327 17604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:26.264528 17604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:26.264729 17604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:26.265021 17604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:26.265592 17656 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:10:26.267920 17604 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:10:26.268015 17604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260808 10:10:26.268071 17604 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:10:26.268934 17604 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:10:26.269003 17604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 10:10:26.269119 17656 tablet_bootstrap.cc:492] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c: Bootstrap starting.
I20260808 10:10:26.277915 17604 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:33175
I20260808 10:10:26.278388 17604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:10:26.279119 17769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:33175 every 8 connection(s)
I20260808 10:10:26.282615 17656 log.cc:824] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:26.285724 17656 tablet_bootstrap.cc:492] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:10:26.285940 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 17604
I20260808 10:10:26.286100 17656 tablet_bootstrap.cc:492] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c: Bootstrap complete.
I20260808 10:10:26.286305 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:35479
--local_ip_for_outbound_sockets=127.2.30.130
--tserver_master_addrs=127.2.30.190:36255
--webserver_port=44741
--webserver_interface=127.2.30.130
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:10:26.287885 17656 ts_tablet_manager.cc:1397] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.008s
I20260808 10:10:26.289572 17656 raft_consensus.cc:348] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:26.290004 17656 raft_consensus.cc:729] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf56168db55404981e1183fb713760c, State: Initialized, Role: FOLLOWER
I20260808 10:10:26.290197 17656 consensus_queue.cc:260] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:26.290505 17656 ts_tablet_manager.cc:1428] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
W20260808 10:10:26.373842 17773 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:10:26.279348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:45539)
0808 10:10:26.282971 (+ 3623us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:26.283069 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:26.283777 (+ 708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:26.373587 (+ 89810us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":3549,"thread_start_us":104,"threads_started":1}
W20260808 10:10:26.374171 17770 heartbeater.cc:644] Failed to heartbeat to 127.2.30.190:36255 (0 consecutive failures): Network error: Failed to ping master at 127.2.30.190:36255: Client connection negotiation failed: client connection to 127.2.30.190:36255: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:10:26.374588 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:26.374462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:35497)
0808 10:10:26.374516 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:26.374557 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 10:10:26.375043 17773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:26.374899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:56197)
0808 10:10:26.374947 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:26.375022 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 10:10:26.375186 17770 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:10:26.391762 17775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:26.392004 17775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:26.392031 17775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:26.393345 17775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:26.393409 17775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:26.393437 17775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:10:26.395164 17775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:35479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=44741
--tserver_master_addrs=127.2.30.190:36255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:26.395455 17775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:26.395735 17775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:26.401487 17782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:26.401520 17785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:26.401557 17775 server_base.cc:1034] running on GCE node
W20260808 10:10:26.401520 17783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:26.402009 17775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:26.402449 17775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:26.403643 17775 hybrid_clock.cc:648] HybridClock initialized: now 1786183826403615 us; error 45 us; skew 500 ppm
Aug 08 10:10:26 dist-test-slave-w206 krb5kdc[16425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183826, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:26.406692 17775 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:10:26.407295 17775 webserver.cc:460] Webserver started at http://127.2.30.130:44741/ using document root <none> and password file <none>
I20260808 10:10:26.407586 17775 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:26.407640 17775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:26.409264 17775 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:10:26.410013 17792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:26.410244 17775 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:26.410351 17775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "c8b8c6016b1648cab00c272f178a0137"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "3a7035de113beda7dde2c15de00f50cf"
server_key_iv: "2e1b074a7cdb6b59915a01e0fb6ad080"
server_key_version: "encryptionkey@0"
I20260808 10:10:26.410728 17775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:26.420861 17775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:26.423605 17775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:26.423739 17775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:26.423978 17775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:26.424345 17775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:10:26.424382 17775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:26.424407 17775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:10:26.424419 17775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:26.431135 17775 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:35479
I20260808 10:10:26.431376 17908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:35479 every 8 connection(s)
I20260808 10:10:26.431702 17775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:10:26.433179 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:26.432499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:48747)
0808 10:10:26.432940 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:26.433022 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":365,"threads_started":1}
W20260808 10:10:26.433836 17909 heartbeater.cc:644] Failed to heartbeat to 127.2.30.190:36255 (0 consecutive failures): Network error: Failed to ping master at 127.2.30.190:36255: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
I20260808 10:10:26.434194 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:26.434079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:40981)
0808 10:10:26.434142 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:26.434172 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 10:10:26.434580 17911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:26.434450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:48509)
0808 10:10:26.434527 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:26.434558 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 10:10:26.434739 17909 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:10:26.435510 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 17775
I20260808 10:10:26.435729 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:37833
--local_ip_for_outbound_sockets=127.2.30.131
--tserver_master_addrs=127.2.30.190:36255
--webserver_port=34205
--webserver_interface=127.2.30.131
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:26.525027 17912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:26.525276 17912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:26.525305 17912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:26.526533 17912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:26.526580 17912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:26.526609 17912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:10:26.528306 17912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:37833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=34205
--tserver_master_addrs=127.2.30.190:36255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:26.528544 17912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:26.528779 17912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:10:26.535065 17912 server_base.cc:1034] running on GCE node
W20260808 10:10:26.535005 17918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:26.535089 17921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:26.535032 17917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:26.535421 17912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:26.535719 17912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:26.536916 17912 hybrid_clock.cc:648] HybridClock initialized: now 1786183826536886 us; error 44 us; skew 500 ppm
Aug 08 10:10:26 dist-test-slave-w206 krb5kdc[16425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183826, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:26.539882 17912 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:10:26.540370 17912 webserver.cc:460] Webserver started at http://127.2.30.131:34205/ using document root <none> and password file <none>
I20260808 10:10:26.540649 17912 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:26.540700 17912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:26.542198 17912 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 10:10:26.543289 17929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:26.543541 17912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:10:26.543658 17912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "e586a293ae614b3f8a1b77faf45c331b"
format_stamp: "Formatted at 2026-08-08 10:10:24 on dist-test-slave-w206"
server_key: "1f3377ed638db8ed602b1fb38b643ed9"
server_key_iv: "773ff875570ce051703126f6272d074f"
server_key_version: "encryptionkey@0"
I20260808 10:10:26.544145 17912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:26.587597 17912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:26.590044 17912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:26.590237 17912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:26.590583 17912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:26.590927 17912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:10:26.590984 17912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:26.591073 17912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:10:26.591104 17912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:26.598606 17912 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:37833
I20260808 10:10:26.598655 18047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:37833 every 8 connection(s)
I20260808 10:10:26.599193 17912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:10:26.600634 18050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:26.600139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:48315)
0808 10:10:26.600368 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:26.600456 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":103,"threads_started":1}
W20260808 10:10:26.601223 18048 heartbeater.cc:644] Failed to heartbeat to 127.2.30.190:36255 (0 consecutive failures): Network error: Failed to ping master at 127.2.30.190:36255: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
I20260808 10:10:26.601639 18050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:26.601494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:39553)
0808 10:10:26.601567 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:26.601603 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 10:10:26.602082 18050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:26.601961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:56037)
0808 10:10:26.602026 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:26.602058 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 10:10:26.602260 18048 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:10:26.603209 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 17912
I20260808 10:10:27.375994 18052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:27.375686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:37389)
0808 10:10:27.375872 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:27.375920 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":82,"threads_started":1}
I20260808 10:10:27.435338 18053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:27.435109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:51823)
0808 10:10:27.435241 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:27.435275 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
I20260808 10:10:27.603073 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:27.602704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:40529)
0808 10:10:27.602893 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:27.602932 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":124,"threads_started":1}
I20260808 10:10:27.711669 18055 raft_consensus.cc:482] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:10:27.711826 18055 raft_consensus.cc:504] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:27.712251 18055 leader_election.cc:288] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833), c8b8c6016b1648cab00c272f178a0137 (127.2.30.130:35479)
Aug 08 10:10:27 dist-test-slave-w206 krb5kdc[16425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183826, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.131@KRBTEST.COM
Aug 08 10:10:27 dist-test-slave-w206 krb5kdc[16425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183826, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.130@KRBTEST.COM
I20260808 10:10:27.724767 18058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:27.712898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:60553 (local address 127.2.30.130:35479)
0808 10:10:27.713108 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:27.713122 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:27.713227 (+ 105us) server_negotiation.cc:407] Connection header received
0808 10:10:27.714040 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:27.714044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:27.714096 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:27.714236 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:27.715117 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:27.715999 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:27.717095 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:27.717511 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:27.717630 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:27.720761 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:27.720767 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:27.720768 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:27.720783 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:27.722801 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:27.723431 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:27.723434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:27.723434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:27.723548 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:27.723895 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:27.723896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:27.723896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:27.724174 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:27.724257 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:27.724520 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 10:10:27.724654 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":71,"threads_started":1}
I20260808 10:10:27.724754 18052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:27.712440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:37833 (local address 127.2.30.129:59609)
0808 10:10:27.712491 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:27.712506 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:27.712613 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:27.714122 (+ 1509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:27.714126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:27.714645 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:27.714925 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:27.714932 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:27.716143 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:27.716144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:27.716995 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:27.716998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:27.717605 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:27.717606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:27.717734 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:27.718417 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:27.718423 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:27.720324 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:27.722653 (+ 2329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:27.722664 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:27.722667 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:27.723124 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:27.723430 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:27.723431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:27.723431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:27.723584 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:27.724165 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:27.724173 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:27.724328 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 10:10:27.724422 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 10:10:27.724640 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":50}
I20260808 10:10:27.724846 18057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:27.712728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:59609 (local address 127.2.30.131:37833)
0808 10:10:27.713185 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:27.713195 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:27.713265 (+ 70us) server_negotiation.cc:407] Connection header received
0808 10:10:27.713826 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:27.713829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:27.713931 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:27.714132 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:27.715097 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:27.716032 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:27.717090 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:27.717508 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:27.717630 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:27.720507 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:27.720513 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:27.720513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:27.720534 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:27.722533 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:27.723221 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:27.723225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:27.723226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:27.723357 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:27.723753 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:27.723755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:27.723755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:27.724084 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:27.724152 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:27.724521 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 10:10:27.724718 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":63,"threads_started":1}
I20260808 10:10:27.725445 18059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:27.712440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:35479 (local address 127.2.30.129:60553)
0808 10:10:27.713130 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:27.713143 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:27.713262 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:27.714359 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:27.714360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:27.714661 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:27.714926 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:27.714933 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:27.716125 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:27.716126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:27.716995 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:27.716998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:27.717610 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:27.717611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:27.717734 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:27.718413 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:27.718422 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:27.720639 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:27.722949 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:27.722951 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:27.722951 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:27.723340 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:27.723704 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:27.723705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:27.723706 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:27.723812 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:27.724258 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:27.724259 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:27.724327 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 10:10:27.724443 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 10:10:27.724729 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":755,"thread_start_us":72,"threads_started":1}
W20260808 10:10:27.725680 17650 leader_election.cc:341] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833): Not found: Tablet not found: a7b44a90fb4b4eefb0deb99d9fbb717e
W20260808 10:10:27.726126 17651 leader_election.cc:341] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8b8c6016b1648cab00c272f178a0137 (127.2.30.130:35479): Not found: Tablet not found: a7b44a90fb4b4eefb0deb99d9fbb717e
I20260808 10:10:27.726203 17651 leader_election.cc:302] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdf56168db55404981e1183fb713760c; no voters: c8b8c6016b1648cab00c272f178a0137, e586a293ae614b3f8a1b77faf45c331b
I20260808 10:10:27.726310 18055 raft_consensus.cc:2726] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:10:28.376914 18060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:28.376633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:53251)
0808 10:10:28.376812 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:28.376862 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":79,"threads_started":1}
I20260808 10:10:28.436305 18061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:28.435986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:57133)
0808 10:10:28.436205 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:28.436254 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":65,"threads_started":1}
I20260808 10:10:28.603902 18062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:28.603615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:54897)
0808 10:10:28.603803 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:28.603839 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":86,"threads_started":1}
I20260808 10:10:29.377741 18063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:29.377463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:39873)
0808 10:10:29.377628 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:29.377688 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":69,"threads_started":1}
I20260808 10:10:29.437207 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:29.436912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:44717)
0808 10:10:29.437103 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:29.437152 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":87,"threads_started":1}
I20260808 10:10:29.604681 18065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:29.604421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:33729)
0808 10:10:29.604584 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:29.604630 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":71,"threads_started":1}
I20260808 10:10:29.794380 18066 raft_consensus.cc:482] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:10:29.794521 18066 raft_consensus.cc:504] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:29.794817 18066 leader_election.cc:288] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833), c8b8c6016b1648cab00c272f178a0137 (127.2.30.130:35479)
W20260808 10:10:29.795400 17651 leader_election.cc:341] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8b8c6016b1648cab00c272f178a0137 (127.2.30.130:35479): Not found: Tablet not found: a7b44a90fb4b4eefb0deb99d9fbb717e
W20260808 10:10:29.795481 17650 leader_election.cc:341] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833): Not found: Tablet not found: a7b44a90fb4b4eefb0deb99d9fbb717e
I20260808 10:10:29.795600 17650 leader_election.cc:302] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdf56168db55404981e1183fb713760c; no voters: c8b8c6016b1648cab00c272f178a0137, e586a293ae614b3f8a1b77faf45c331b
I20260808 10:10:29.795680 18066 raft_consensus.cc:2726] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:10:30.378885 18067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:30.378483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:60075)
0808 10:10:30.378761 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:30.378819 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":104,"threads_started":1}
I20260808 10:10:30.438097 18068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:30.437823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:39807)
0808 10:10:30.438001 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:30.438048 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
I20260808 10:10:30.605598 18069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:30.605312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:56127)
0808 10:10:30.605487 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:30.605541 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":68,"threads_started":1}
I20260808 10:10:31.379884 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:31.379560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:53979)
0808 10:10:31.379733 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:31.379828 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":86,"threads_started":1}
I20260808 10:10:31.438948 18071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:31.438668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:39105)
0808 10:10:31.438856 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:31.438899 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":93,"threads_started":1}
I20260808 10:10:31.606536 18072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:31.606225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:37905)
0808 10:10:31.606431 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:31.606483 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":117,"threads_started":1}
I20260808 10:10:32.201964 18073 raft_consensus.cc:482] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:10:32.202072 18073 raft_consensus.cc:504] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:32.202334 18073 leader_election.cc:288] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833), c8b8c6016b1648cab00c272f178a0137 (127.2.30.130:35479)
W20260808 10:10:32.202813 17650 leader_election.cc:341] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833): Not found: Tablet not found: a7b44a90fb4b4eefb0deb99d9fbb717e
W20260808 10:10:32.202913 17651 leader_election.cc:341] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8b8c6016b1648cab00c272f178a0137 (127.2.30.130:35479): Not found: Tablet not found: a7b44a90fb4b4eefb0deb99d9fbb717e
I20260808 10:10:32.203001 17651 leader_election.cc:302] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdf56168db55404981e1183fb713760c; no voters: c8b8c6016b1648cab00c272f178a0137, e586a293ae614b3f8a1b77faf45c331b
I20260808 10:10:32.203094 18073 raft_consensus.cc:2726] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:10:32.380859 18075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:32.380564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:52525)
0808 10:10:32.380741 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:32.380789 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":94,"threads_started":1}
I20260808 10:10:32.439783 18076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:32.439535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:38679)
0808 10:10:32.439684 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:32.439723 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":66,"threads_started":1}
I20260808 10:10:32.607611 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:32.607349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:43249)
0808 10:10:32.607511 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:32.607557 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":76,"threads_started":1}
I20260808 10:10:33.381707 18078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:33.381481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:40307)
0808 10:10:33.381612 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:33.381658 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260808 10:10:33.440690 18079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:33.440386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:44335)
0808 10:10:33.440553 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:33.440606 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":74,"threads_started":1}
I20260808 10:10:33.608500 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:33.608234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:60411)
0808 10:10:33.608397 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:33.608445 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":71,"threads_started":1}
I20260808 10:10:34.382593 18082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:34.382322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:57023)
0808 10:10:34.382495 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:34.382542 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":82,"threads_started":1}
I20260808 10:10:34.441565 18083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:34.441293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:33149)
0808 10:10:34.441458 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:34.441508 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
I20260808 10:10:34.609442 18084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:34.609165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:52211)
0808 10:10:34.609336 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:34.609385 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":82,"threads_started":1}
I20260808 10:10:35.120244 18085 raft_consensus.cc:482] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:10:35.120360 18085 raft_consensus.cc:504] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:35.120558 18085 leader_election.cc:288] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833), c8b8c6016b1648cab00c272f178a0137 (127.2.30.130:35479)
W20260808 10:10:35.121052 17651 leader_election.cc:341] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8b8c6016b1648cab00c272f178a0137 (127.2.30.130:35479): Not found: Tablet not found: a7b44a90fb4b4eefb0deb99d9fbb717e
W20260808 10:10:35.121162 17650 leader_election.cc:341] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833): Not found: Tablet not found: a7b44a90fb4b4eefb0deb99d9fbb717e
I20260808 10:10:35.121233 17650 leader_election.cc:302] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdf56168db55404981e1183fb713760c; no voters: c8b8c6016b1648cab00c272f178a0137, e586a293ae614b3f8a1b77faf45c331b
I20260808 10:10:35.121356 18085 raft_consensus.cc:2726] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:10:35.383562 18086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:35.383287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:52935)
0808 10:10:35.383461 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:35.383509 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":77,"threads_started":1}
I20260808 10:10:35.442610 18087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:35.442241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:55165)
0808 10:10:35.442494 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:35.442545 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":148,"threads_started":1}
I20260808 10:10:35.610345 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:35.610101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:41627)
0808 10:10:35.610240 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:35.610285 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260808 10:10:36.384732 18089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:36.384326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:57411)
0808 10:10:36.384591 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:36.384644 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":208,"threads_started":1}
I20260808 10:10:36.443562 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:36.443234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:33517)
0808 10:10:36.443466 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:36.443509 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":116,"threads_started":1}
I20260808 10:10:36.611326 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:36.610965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:57905)
0808 10:10:36.611226 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:36.611275 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":72,"threads_started":1}
I20260808 10:10:37.385808 18092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:37.385495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:48873)
0808 10:10:37.385683 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:37.385756 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":74,"threads_started":1}
I20260808 10:10:37.444509 18093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:37.444163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:40445)
0808 10:10:37.444378 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:37.444436 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":94,"threads_started":1}
I20260808 10:10:37.612604 18094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:37.612162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:49003)
0808 10:10:37.612500 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:37.612551 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":80,"threads_started":1}
I20260808 10:10:38.386883 18095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:38.386573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:38047)
0808 10:10:38.386762 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:38.386814 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":75,"threads_started":1}
I20260808 10:10:38.445374 18096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:38.445021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:57341)
0808 10:10:38.445272 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:38.445321 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":67,"threads_started":1}
I20260808 10:10:38.613603 18097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:38.613345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:35923)
0808 10:10:38.613502 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:38.613544 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":105,"threads_started":1}
I20260808 10:10:39.387988 18098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:39.387621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:58749)
0808 10:10:39.387823 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:39.387909 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":115,"threads_started":1}
I20260808 10:10:39.446291 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:39.445978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:49889)
0808 10:10:39.446182 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:39.446240 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":87,"threads_started":1}
I20260808 10:10:39.614535 18100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:39.614240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:55399)
0808 10:10:39.614414 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:39.614463 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":100,"threads_started":1}
I20260808 10:10:40.388792 18101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:40.388499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:51279)
0808 10:10:40.388677 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:40.388726 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":77,"threads_started":1}
I20260808 10:10:40.447207 18102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:40.446858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:58129)
0808 10:10:40.447061 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:40.447115 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":84,"threads_started":1}
I20260808 10:10:40.546954 18103 raft_consensus.cc:482] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:10:40.547144 18103 raft_consensus.cc:504] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e586a293ae614b3f8a1b77faf45c331b" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37833 } } peers { permanent_uuid: "c8b8c6016b1648cab00c272f178a0137" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35479 } } peers { permanent_uuid: "bdf56168db55404981e1183fb713760c" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 33175 } }
I20260808 10:10:40.547381 18103 leader_election.cc:288] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833), c8b8c6016b1648cab00c272f178a0137 (127.2.30.130:35479)
W20260808 10:10:40.547951 17651 leader_election.cc:341] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c8b8c6016b1648cab00c272f178a0137 (127.2.30.130:35479): Not found: Tablet not found: a7b44a90fb4b4eefb0deb99d9fbb717e
W20260808 10:10:40.548041 17650 leader_election.cc:341] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e586a293ae614b3f8a1b77faf45c331b (127.2.30.131:37833): Not found: Tablet not found: a7b44a90fb4b4eefb0deb99d9fbb717e
I20260808 10:10:40.548071 17651 leader_election.cc:302] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdf56168db55404981e1183fb713760c; no voters: c8b8c6016b1648cab00c272f178a0137, e586a293ae614b3f8a1b77faf45c331b
I20260808 10:10:40.548149 18103 raft_consensus.cc:2726] T a7b44a90fb4b4eefb0deb99d9fbb717e P bdf56168db55404981e1183fb713760c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:10:40.615473 18104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:40.615205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:55537)
0808 10:10:40.615370 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:40.615412 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260808 10:10:41.389909 18105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:41.389595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.129:49597)
0808 10:10:41.389816 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:41.389859 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":72,"threads_started":1}
I20260808 10:10:41.448232 18106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:41.447943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.130:60429)
0808 10:10:41.448088 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:41.448168 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260808 10:10:41.609922 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 17604
I20260808 10:10:41.616478 18107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:41.616230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36255 (local address 127.2.30.131:52915)
0808 10:10:41.616378 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:41.616430 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:36255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260808 10:10:41.617841 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 17775
I20260808 10:10:41.624639 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 17912
2026-08-08T10:10:41Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18682 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:10:41 dist-test-slave-w206 krb5kdc[18110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:10:41 dist-test-slave-w206 krb5kdc[18110](info): set up 2 sockets
Aug 08 10:10:41 dist-test-slave-w206 krb5kdc[18110](info): commencing operation
krb5kdc: starting...
W20260808 10:10:42.773577 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:10:42 dist-test-slave-w206 krb5kdc[18110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:10:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:10:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:10:42.930013 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38189
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:39861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:38189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:43.031391 18123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:43.031701 18123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:43.031744 18123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:43.033037 18123 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 10:10:43.033133 18123 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 10:10:43.033161 18123 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 10:10:43.033223 18123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:10:43.033258 18123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:43.033290 18123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:43.033318 18123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:10:43.033344 18123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:10:43.035121 18123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:38189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:43.035434 18123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:43.035699 18123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:10:43.041774 18123 server_base.cc:1034] running on GCE node
W20260808 10:10:43.041695 18131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:43.041728 18128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:43.041857 18129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:43.042476 18123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:43.042801 18123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:43.043977 18123 hybrid_clock.cc:648] HybridClock initialized: now 1786183843043957 us; error 40 us; skew 500 ppm
Aug 08 10:10:43 dist-test-slave-w206 krb5kdc[18110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183843, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:43.047535 18123 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:10:43.048134 18123 webserver.cc:460] Webserver started at http://127.2.30.190:34793/ using document root <none> and password file <none>
I20260808 10:10:43.048506 18123 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:43.048585 18123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:43.048744 18123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:10:43.049965 18123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "7102975c4a3946a5a00f9b9837293022"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "86d2e5f26b9f05b2afa08e167e9d3aec"
server_key_iv: "6f86eb2343a28d0bfd37688789d36c79"
server_key_version: "encryptionkey@0"
I20260808 10:10:43.050386 18123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "7102975c4a3946a5a00f9b9837293022"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "86d2e5f26b9f05b2afa08e167e9d3aec"
server_key_iv: "6f86eb2343a28d0bfd37688789d36c79"
server_key_version: "encryptionkey@0"
I20260808 10:10:43.052059 18123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 10:10:43.053097 18138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:43.053310 18123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:10:43.053488 18123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "7102975c4a3946a5a00f9b9837293022"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "86d2e5f26b9f05b2afa08e167e9d3aec"
server_key_iv: "6f86eb2343a28d0bfd37688789d36c79"
server_key_version: "encryptionkey@0"
I20260808 10:10:43.053573 18123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:43.073202 18123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:43.076166 18123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:43.076472 18123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:43.081988 18123 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:38189
I20260808 10:10:43.082031 18192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:38189 every 8 connection(s)
I20260808 10:10:43.082465 18123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:10:43.083503 18193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:43.086066 18193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Bootstrap starting.
I20260808 10:10:43.086804 18193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:43.087190 18193 log.cc:824] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:43.087697 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 18123
I20260808 10:10:43.087793 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:10:43.088048 2170 external_mini_cluster.cc:1428] Setting key acf8cfd841b52f98858aa43c54b710c6
I20260808 10:10:43.088274 18193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: No bootstrap required, opened a new log
I20260808 10:10:43.091396 18193 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } }
I20260808 10:10:43.091629 18193 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:43.091683 18193 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7102975c4a3946a5a00f9b9837293022, State: Initialized, Role: FOLLOWER
I20260808 10:10:43.091874 18193 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } }
I20260808 10:10:43.092006 18193 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:10:43.092057 18193 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:10:43.092113 18193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:43.092922 18193 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } }
I20260808 10:10:43.093062 18193 leader_election.cc:302] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7102975c4a3946a5a00f9b9837293022; no voters:
I20260808 10:10:43.093281 18193 leader_election.cc:288] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:10:43.093358 18198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:10:43.093583 18198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 1 LEADER]: Becoming Leader. State: Replica: 7102975c4a3946a5a00f9b9837293022, State: Running, Role: LEADER
I20260808 10:10:43.093683 18193 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:10:43.093845 18198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } }
I20260808 10:10:43.094357 18200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7102975c4a3946a5a00f9b9837293022. Latest consensus state: current_term: 1 leader_uuid: "7102975c4a3946a5a00f9b9837293022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } } }
I20260808 10:10:43.094393 18199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7102975c4a3946a5a00f9b9837293022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } } }
I20260808 10:10:43.094537 18199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [sys.catalog]: This master's current role is: LEADER
I20260808 10:10:43.094810 18200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [sys.catalog]: This master's current role is: LEADER
I20260808 10:10:43.094821 18208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:10:43.097215 18208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 10:10:43 dist-test-slave-w206 krb5kdc[18110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183842, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:43.099653 18208 catalog_manager.cc:1347] Generated new cluster ID: 3999e8f231ed49bd8d5f6647b2b0f3fa
I20260808 10:10:43.099818 18208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:10:43.102916 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.088766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52322 (local address 127.2.30.190:38189)
0808 10:10:43.089012 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:43.089030 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:43.089059 (+ 29us) server_negotiation.cc:407] Connection header received
0808 10:10:43.089974 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:43.089990 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:43.090049 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:43.090180 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:43.091275 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.092769 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.094051 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.094789 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.094922 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.097989 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:43.097994 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:43.097995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:43.098010 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:43.100994 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:43.101448 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:43.101451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:43.101451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:43.101589 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:43.101883 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:43.101884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:43.101884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:43.102204 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:43.102384 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:43.102485 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 10:10:43.102655 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":138,"threads_started":1}
I20260808 10:10:43.114049 18208 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:10:43.114741 18208 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:10:43.122323 18208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Generated new TSK 0
I20260808 10:10:43.122560 18208 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:10:43.171970 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38189
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:39861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:43.307365 18221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:43.307703 18221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:43.307785 18221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:43.309217 18221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:43.309305 18221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:43.309384 18221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:10:43.311300 18221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:43.311566 18221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:43.311930 18221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:43.317925 18226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:43.317996 18229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:43.317925 18227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:43.318251 18221 server_base.cc:1034] running on GCE node
I20260808 10:10:43.318490 18221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:43.318821 18221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:43.320055 18221 hybrid_clock.cc:648] HybridClock initialized: now 1786183843320049 us; error 76 us; skew 500 ppm
Aug 08 10:10:43 dist-test-slave-w206 krb5kdc[18110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183843, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:43.323052 18221 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:10:43.323794 18221 webserver.cc:460] Webserver started at http://127.2.30.129:34293/ using document root <none> and password file <none>
I20260808 10:10:43.324112 18221 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:43.324224 18221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:43.324425 18221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:10:43.325907 18221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "1635c9eb26f14513b0ec873c65662bad"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "d751ffd5da5b33481a9b2f7db24a73fb"
server_key_iv: "63621ec6b71fc1e7781dffd591b6d0ac"
server_key_version: "encryptionkey@0"
I20260808 10:10:43.326398 18221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "1635c9eb26f14513b0ec873c65662bad"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "d751ffd5da5b33481a9b2f7db24a73fb"
server_key_iv: "63621ec6b71fc1e7781dffd591b6d0ac"
server_key_version: "encryptionkey@0"
I20260808 10:10:43.328140 18221 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 10:10:43.329380 18236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:43.329695 18221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:43.329847 18221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "1635c9eb26f14513b0ec873c65662bad"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "d751ffd5da5b33481a9b2f7db24a73fb"
server_key_iv: "63621ec6b71fc1e7781dffd591b6d0ac"
server_key_version: "encryptionkey@0"
I20260808 10:10:43.329936 18221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:43.348905 18221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:43.351362 18221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:43.351526 18221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:43.351747 18221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:43.352126 18221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:10:43.352191 18221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:43.352229 18221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:10:43.352272 18221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:43.360260 18221 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:44791
I20260808 10:10:43.360350 18353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:44791 every 8 connection(s)
I20260808 10:10:43.360839 18221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:10:43.370847 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 18221
I20260808 10:10:43.370966 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:10:43.371315 2170 external_mini_cluster.cc:1428] Setting key fd7bd5fff071196230b10557986059d1
Aug 08 10:10:43 dist-test-slave-w206 krb5kdc[18110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183843, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:43.376237 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.361793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:37411 (local address 127.2.30.190:38189)
0808 10:10:43.362017 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:43.362019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:43.362033 (+ 14us) server_negotiation.cc:407] Connection header received
0808 10:10:43.362618 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:43.362619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:43.362632 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:43.362703 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:43.364363 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.366413 (+ 2050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.368890 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.369113 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.369185 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.372165 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:43.372166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:43.372166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:43.372174 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:43.374158 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:43.374955 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:43.374956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:43.374956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:43.375086 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:43.375429 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:43.375430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:43.375430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:43.375556 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:43.375609 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:43.376032 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 10:10:43.376150 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260808 10:10:43.376576 18356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.361671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:37411)
0808 10:10:43.361899 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:43.361947 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:43.362496 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:43.362786 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:43.362790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:43.363559 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:43.364145 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:43.364151 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.366967 (+ 2816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.367096 (+ 129us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:43.368719 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:43.368733 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.369217 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.369218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:43.369346 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.370023 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:43.370032 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:43.372031 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:43.374324 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:43.374338 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:43.374341 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:43.374842 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:43.375160 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:43.375161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:43.375162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:43.375346 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:43.375632 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:43.375640 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:43.375881 (+ 241us) client_negotiation.cc:769] Sending connection context
0808 10:10:43.375997 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 10:10:43.376273 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":129,"threads_started":1}
I20260808 10:10:43.377233 18354 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38189
I20260808 10:10:43.377365 18354 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:43.377585 18354 heartbeater.cc:511] Master 127.2.30.190:38189 requested a full tablet report, sending...
I20260808 10:10:43.378217 18156 ts_manager.cc:194] Registered new tserver with Master: 1635c9eb26f14513b0ec873c65662bad (127.2.30.129:44791)
I20260808 10:10:43.378880 18156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:37411
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:10:43.407250 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38189
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:39861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:43.508356 18359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:43.508623 18359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:43.508654 18359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:43.510037 18359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:43.510118 18359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:43.510165 18359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:10:43.512087 18359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:43.512409 18359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:43.512687 18359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:43.519258 18364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:43.519234 18359 server_base.cc:1034] running on GCE node
W20260808 10:10:43.519234 18367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:43.519305 18365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:43.519775 18359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:43.520116 18359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:43.521307 18359 hybrid_clock.cc:648] HybridClock initialized: now 1786183843521278 us; error 52 us; skew 500 ppm
Aug 08 10:10:43 dist-test-slave-w206 krb5kdc[18110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183843, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:43.530184 18359 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:10:43.530754 18359 webserver.cc:460] Webserver started at http://127.2.30.130:33171/ using document root <none> and password file <none>
I20260808 10:10:43.531119 18359 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:43.531194 18359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:43.531597 18359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:10:43.532898 18359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "379856a2bbad46dbaee75d1d406a1514"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "259c2784db517bc7bb66994448a68394"
server_key_iv: "801b17b01b979473fd1ea23c90182e28"
server_key_version: "encryptionkey@0"
I20260808 10:10:43.533334 18359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "379856a2bbad46dbaee75d1d406a1514"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "259c2784db517bc7bb66994448a68394"
server_key_iv: "801b17b01b979473fd1ea23c90182e28"
server_key_version: "encryptionkey@0"
I20260808 10:10:43.535475 18359 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 10:10:43.536510 18375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:43.536871 18359 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:43.537010 18359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "379856a2bbad46dbaee75d1d406a1514"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "259c2784db517bc7bb66994448a68394"
server_key_iv: "801b17b01b979473fd1ea23c90182e28"
server_key_version: "encryptionkey@0"
I20260808 10:10:43.537107 18359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:43.552634 18359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:43.555356 18359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:43.555552 18359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:43.555853 18359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:43.556211 18359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:10:43.556277 18359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:43.556360 18359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:10:43.556389 18359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:43.564043 18359 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:36317
I20260808 10:10:43.564085 18491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:36317 every 8 connection(s)
I20260808 10:10:43.564464 18359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:10:43.565553 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 18359
I20260808 10:10:43.565716 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:10:43.565937 2170 external_mini_cluster.cc:1428] Setting key 0fb60daef17b51ed914cb36e628ca9be
Aug 08 10:10:43 dist-test-slave-w206 krb5kdc[18110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183843, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
I20260808 10:10:43.577148 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.565355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:47439 (local address 127.2.30.190:38189)
0808 10:10:43.565424 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:43.565426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:43.565638 (+ 212us) server_negotiation.cc:407] Connection header received
0808 10:10:43.566273 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:43.566274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:43.566286 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:43.566382 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:43.567543 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.568388 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.569627 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.569855 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.569940 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.573053 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:43.573055 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:43.573056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:43.573072 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:43.574862 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:43.575764 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:43.575766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:43.575766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:43.575839 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:43.576254 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:43.576255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:43.576256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:43.576411 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:43.576514 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:43.576913 (+ 399us) server_negotiation.cc:299] Negotiation successful
0808 10:10:43.577021 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 10:10:43.577390 18495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.565229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:47439)
0808 10:10:43.565481 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:43.565532 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:43.566159 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:43.566448 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:43.566451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:43.567218 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:43.567429 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:43.567435 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.568530 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.568531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:43.569491 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:43.569495 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.569953 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.569954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:43.570108 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.571304 (+ 1196us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:43.571315 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:43.572926 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:43.575034 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:43.575046 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:43.575049 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:43.575663 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:43.575947 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:43.575958 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:43.575958 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:43.576122 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:43.576514 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:43.576522 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:43.576729 (+ 207us) client_negotiation.cc:769] Sending connection context
0808 10:10:43.576869 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 10:10:43.577085 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":109,"threads_started":1}
I20260808 10:10:43.578043 18493 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38189
I20260808 10:10:43.578197 18493 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:43.578506 18493 heartbeater.cc:511] Master 127.2.30.190:38189 requested a full tablet report, sending...
I20260808 10:10:43.579113 18156 ts_manager.cc:194] Registered new tserver with Master: 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317)
I20260808 10:10:43.579551 18156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:47439
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:10:43.603564 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38189
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:39861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:10:43.708899 18498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:10:43.709311 18498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:10:43.709364 18498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:10:43.710889 18498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:10:43.710973 18498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:10:43.711067 18498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:10:43.713714 18498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:10:43.714102 18498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:10:43.714533 18498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:10:43.722126 18506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:43.722103 18504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:10:43.722265 18503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:10:43.723344 18498 server_base.cc:1034] running on GCE node
I20260808 10:10:43.723631 18498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:10:43.724013 18498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:10:43.725265 18498 hybrid_clock.cc:648] HybridClock initialized: now 1786183843725250 us; error 32 us; skew 500 ppm
Aug 08 10:10:43 dist-test-slave-w206 krb5kdc[18110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183843, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:10:43.729732 18498 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:10:43.730628 18498 webserver.cc:460] Webserver started at http://127.2.30.131:37433/ using document root <none> and password file <none>
I20260808 10:10:43.731045 18498 fs_manager.cc:359] Metadata directory not provided
I20260808 10:10:43.731119 18498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:10:43.731302 18498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:10:43.732733 18498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "b896e6cb7fa74813a48e87cd76098119"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "fb98b0dc6592c56fee9441b659c543d1"
server_key_iv: "53b4d90ed7cc6f94d830e71c7c6edc8c"
server_key_version: "encryptionkey@0"
I20260808 10:10:43.733182 18498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "b896e6cb7fa74813a48e87cd76098119"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "fb98b0dc6592c56fee9441b659c543d1"
server_key_iv: "53b4d90ed7cc6f94d830e71c7c6edc8c"
server_key_version: "encryptionkey@0"
I20260808 10:10:43.735436 18498 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 10:10:43.736555 18513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:43.737056 18498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:10:43.737198 18498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "b896e6cb7fa74813a48e87cd76098119"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "fb98b0dc6592c56fee9441b659c543d1"
server_key_iv: "53b4d90ed7cc6f94d830e71c7c6edc8c"
server_key_version: "encryptionkey@0"
I20260808 10:10:43.737404 18498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:10:43.758701 18498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:10:43.762823 18498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:10:43.763059 18498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:10:43.763412 18498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:10:43.763893 18498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:10:43.763935 18498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:43.763959 18498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:10:43.763972 18498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:10:43.771957 18498 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:35927
I20260808 10:10:43.772058 18632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:35927 every 8 connection(s)
I20260808 10:10:43.772432 18498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
Aug 08 10:10:43 dist-test-slave-w206 krb5kdc[18110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183843, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:10:43.782402 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 18498
I20260808 10:10:43.782543 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:10:43.782738 2170 external_mini_cluster.cc:1428] Setting key d1b29af64fb8ef45c4be6b9c73ef69fb
I20260808 10:10:43.784505 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.773290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:55345 (local address 127.2.30.190:38189)
0808 10:10:43.773336 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:43.773338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:43.773579 (+ 241us) server_negotiation.cc:407] Connection header received
0808 10:10:43.774063 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:43.774064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:43.774074 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:43.774143 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:43.775422 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.775973 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.777346 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.777762 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.777840 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.780733 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:43.780735 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:43.780735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:43.780752 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:43.782407 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:43.783152 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:43.783153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:43.783153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:43.783226 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:43.783638 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:43.783639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:43.783640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:43.783803 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:43.783861 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:43.784283 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 10:10:43.784420 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 10:10:43.784660 18635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.773243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:55345)
0808 10:10:43.773428 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:43.773474 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:43.773975 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:43.774202 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:43.774205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:43.775059 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:10:43.775288 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:43.775295 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.776076 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.776077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:43.777095 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:43.777098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.777887 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.777893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:43.778016 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.778772 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:10:43.778782 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:10:43.780611 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:10:43.782561 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:43.782574 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:43.782578 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:43.782982 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:43.783349 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:10:43.783350 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:10:43.783350 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:10:43.783545 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:10:43.783880 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:10:43.783887 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:10:43.784123 (+ 236us) client_negotiation.cc:769] Sending connection context
0808 10:10:43.784230 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 10:10:43.784404 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260808 10:10:43.785236 18633 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38189
I20260808 10:10:43.785364 18633 heartbeater.cc:464] Registering TS with master...
I20260808 10:10:43.785598 18633 heartbeater.cc:511] Master 127.2.30.190:38189 requested a full tablet report, sending...
I20260808 10:10:43.786118 18156 ts_manager.cc:194] Registered new tserver with Master: b896e6cb7fa74813a48e87cd76098119 (127.2.30.131:35927)
I20260808 10:10:43.786621 18156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:55345
I20260808 10:10:43.796079 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:10:43.805611 18197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.798002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52324 (local address 127.2.30.190:38189)
0808 10:10:43.798062 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:43.798065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:43.798077 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:10:43.798145 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:43.798146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:43.798165 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:43.798254 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:10:43.798990 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.799575 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.800523 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.800745 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.800839 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.802565 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:10:43.802568 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:10:43.802568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:10:43.802578 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:10:43.804132 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:43.804621 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:43.804623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:43.804623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:43.804729 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:10:43.804951 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:10:43.804952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:10:43.804952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:10:43.805132 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:10:43.805256 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:43.805369 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 10:10:43.805471 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 10:10:43.807978 18156 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:10:43.810650 18156 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:10:43.819598 18647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.814335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:36317 (local address 127.0.0.1:41050)
0808 10:10:43.814699 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:43.814710 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:43.814774 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:43.815464 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:43.815464 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:43.815466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:43.815510 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:43.815512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.816877 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.816878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:43.818142 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:43.818145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.819192 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.819194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:43.819295 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.819316 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:10:43.819415 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 10:10:43.819493 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":99,"thread_start_us":124,"threads_started":1}
I20260808 10:10:43.819988 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.814493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.2.30.129:44791)
0808 10:10:43.814935 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:43.814941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:43.814952 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:10:43.815015 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:43.815017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:43.815036 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:43.815145 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:43.815360 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.816705 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.818529 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.819332 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.819691 (+ 359us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.819800 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:43.819832 (+ 32us) server_negotiation.cc:299] Negotiation successful
0808 10:10:43.819889 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":97,"threads_started":1}
I20260808 10:10:43.820044 18646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.814295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:44791 (local address 127.0.0.1:41234)
0808 10:10:43.814471 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:43.814513 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:43.814589 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:43.815164 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:43.815168 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:43.815170 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:43.815262 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:43.815264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.816828 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.816829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:43.818406 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:43.818409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.819448 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.819449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:43.819508 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.819513 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:10:43.819546 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 10:10:43.819613 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":171,"thread_start_us":61,"threads_started":1}
I20260808 10:10:43.820123 18648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.814467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41050 (local address 127.2.30.130:36317)
0808 10:10:43.815302 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:43.815308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:43.815318 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:10:43.815371 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:43.815372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:43.815388 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:43.815467 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:43.815575 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.816767 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.818277 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.819090 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.819174 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.819257 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:43.819957 (+ 700us) server_negotiation.cc:299] Negotiation successful
0808 10:10:43.820021 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":795,"thread_start_us":101,"threads_started":1}
I20260808 10:10:43.820655 18650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.814582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:35927 (local address 127.0.0.1:60898)
0808 10:10:43.814716 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:43.814729 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:43.814778 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:43.815422 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:43.815423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:43.815425 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:43.815479 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:43.815482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.816684 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.816685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:43.818414 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:43.818415 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.819401 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.819402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:43.819709 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.819723 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 10:10:43.820524 (+ 801us) client_negotiation.cc:240] Negotiation successful
0808 10:10:43.820586 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":34,"thread_start_us":93,"threads_started":1}
I20260808 10:10:43.820732 18651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.814871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60898 (local address 127.2.30.131:35927)
0808 10:10:43.815123 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:43.815128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:43.815140 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:10:43.815188 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:43.815190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:43.815240 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:43.815334 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:43.815576 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.816545 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.818537 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.819287 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.819394 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.819462 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:43.820578 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0808 10:10:43.820643 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":129,"threads_started":1}
I20260808 10:10:43.821022 18423 tablet_service.cc:1467] Processing CreateTablet for tablet d1a709b6eab8466fb687a7bfae24419b (DEFAULT_TABLE table=test-table [id=6b1b21d4948b471988518cc0f6680fa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:10:43.821489 18423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1a709b6eab8466fb687a7bfae24419b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:43.821587 18562 tablet_service.cc:1467] Processing CreateTablet for tablet d1a709b6eab8466fb687a7bfae24419b (DEFAULT_TABLE table=test-table [id=6b1b21d4948b471988518cc0f6680fa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:10:43.821941 18562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1a709b6eab8466fb687a7bfae24419b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:43.823482 18286 tablet_service.cc:1467] Processing CreateTablet for tablet d1a709b6eab8466fb687a7bfae24419b (DEFAULT_TABLE table=test-table [id=6b1b21d4948b471988518cc0f6680fa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:10:43.823865 18286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1a709b6eab8466fb687a7bfae24419b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:10:43.825155 18652 tablet_bootstrap.cc:492] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119: Bootstrap starting.
I20260808 10:10:43.825335 18653 tablet_bootstrap.cc:492] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514: Bootstrap starting.
I20260808 10:10:43.825927 18652 tablet_bootstrap.cc:654] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:43.826048 18653 tablet_bootstrap.cc:654] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:43.826530 18652 log.cc:824] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:43.826665 18653 log.cc:824] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:43.827631 18653 tablet_bootstrap.cc:492] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514: No bootstrap required, opened a new log
I20260808 10:10:43.827639 18652 tablet_bootstrap.cc:492] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119: No bootstrap required, opened a new log
I20260808 10:10:43.827724 18653 ts_tablet_manager.cc:1397] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:10:43.827741 18652 ts_tablet_manager.cc:1397] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:10:43.827739 18656 tablet_bootstrap.cc:492] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: Bootstrap starting.
I20260808 10:10:43.828328 18656 tablet_bootstrap.cc:654] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: Neither blocks nor log segments found. Creating new log.
I20260808 10:10:43.828824 18656 log.cc:824] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: Log is configured to *not* fsync() on all Append() calls
I20260808 10:10:43.829568 18656 tablet_bootstrap.cc:492] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: No bootstrap required, opened a new log
I20260808 10:10:43.829617 18656 ts_tablet_manager.cc:1397] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:10:43.829922 18652 raft_consensus.cc:348] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:10:43.830119 18652 raft_consensus.cc:374] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:43.830156 18652 raft_consensus.cc:729] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b896e6cb7fa74813a48e87cd76098119, State: Initialized, Role: FOLLOWER
I20260808 10:10:43.830061 18653 raft_consensus.cc:348] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:10:43.830207 18653 raft_consensus.cc:374] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:43.830240 18653 raft_consensus.cc:729] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 379856a2bbad46dbaee75d1d406a1514, State: Initialized, Role: FOLLOWER
I20260808 10:10:43.830302 18652 consensus_queue.cc:260] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:10:43.830332 18653 consensus_queue.cc:260] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:10:43.830616 18652 ts_tablet_manager.cc:1428] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:10:43.830616 18653 ts_tablet_manager.cc:1428] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:10:43.830797 18493 heartbeater.cc:503] Master 127.2.30.190:38189 was elected leader, sending a full tablet report...
I20260808 10:10:43.831066 18633 heartbeater.cc:503] Master 127.2.30.190:38189 was elected leader, sending a full tablet report...
I20260808 10:10:43.831774 18656 raft_consensus.cc:348] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:10:43.831988 18656 raft_consensus.cc:374] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:10:43.832046 18656 raft_consensus.cc:729] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1635c9eb26f14513b0ec873c65662bad, State: Initialized, Role: FOLLOWER
I20260808 10:10:43.832165 18656 consensus_queue.cc:260] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:10:43.832479 18656 ts_tablet_manager.cc:1428] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:10:43.832641 18354 heartbeater.cc:503] Master 127.2.30.190:38189 was elected leader, sending a full tablet report...
W20260808 10:10:43.861873 18355 tablet.cc:2369] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:10:43.934572 18659 raft_consensus.cc:482] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:10:43.934788 18659 raft_consensus.cc:504] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:10:43.935415 18659 leader_election.cc:288] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317), 1635c9eb26f14513b0ec873c65662bad (127.2.30.129:44791)
I20260808 10:10:43.940073 18648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.936002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:55655 (local address 127.2.30.130:36317)
0808 10:10:43.936062 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:43.936065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:43.936205 (+ 140us) server_negotiation.cc:407] Connection header received
0808 10:10:43.936303 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:43.936304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:43.936324 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:43.936396 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:43.936597 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.937118 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.938487 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.939139 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.939229 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.939238 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:43.939888 (+ 650us) server_negotiation.cc:299] Negotiation successful
0808 10:10:43.939969 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 10:10:43.940068 18635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.935794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:44791 (local address 127.2.30.131:36245)
0808 10:10:43.935905 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:43.935923 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:43.936166 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:43.936406 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:43.936407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:43.936409 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:43.936536 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:43.936538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.937221 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.937223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:43.938275 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:43.938280 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.939368 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.939371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:43.939715 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.939723 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:10:43.939822 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 10:10:43.939965 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":98}
I20260808 10:10:43.940068 18649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.935868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:36245 (local address 127.2.30.129:44791)
0808 10:10:43.936070 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:43.936074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:43.936126 (+ 52us) server_negotiation.cc:407] Connection header received
0808 10:10:43.936314 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:43.936315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:43.936327 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:43.936402 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:10:43.936625 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.937106 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.938542 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.939139 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.939229 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.939238 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:43.939909 (+ 671us) server_negotiation.cc:299] Negotiation successful
0808 10:10:43.939961 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:10:43.940403 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.935814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:36317 (local address 127.2.30.131:55655)
0808 10:10:43.936119 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:43.936134 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:10:43.936220 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:10:43.936402 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:10:43.936403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:10:43.936405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:10:43.936496 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:43.936504 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.937224 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.937225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:43.938276 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:10:43.938281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.939372 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.939373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:10:43.939761 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.939769 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 10:10:43.940226 (+ 457us) client_negotiation.cc:240] Negotiation successful
0808 10:10:43.940302 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":6,"thread_start_us":192,"threads_started":1}
I20260808 10:10:43.940778 18307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1a709b6eab8466fb687a7bfae24419b" candidate_uuid: "b896e6cb7fa74813a48e87cd76098119" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1635c9eb26f14513b0ec873c65662bad" is_pre_election: true
I20260808 10:10:43.940987 18444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1a709b6eab8466fb687a7bfae24419b" candidate_uuid: "b896e6cb7fa74813a48e87cd76098119" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379856a2bbad46dbaee75d1d406a1514" is_pre_election: true
I20260808 10:10:43.941046 18307 raft_consensus.cc:2454] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b896e6cb7fa74813a48e87cd76098119 in term 0.
I20260808 10:10:43.941133 18444 raft_consensus.cc:2454] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b896e6cb7fa74813a48e87cd76098119 in term 0.
I20260808 10:10:43.941473 18515 leader_election.cc:302] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1635c9eb26f14513b0ec873c65662bad, b896e6cb7fa74813a48e87cd76098119; no voters:
I20260808 10:10:43.941807 18659 raft_consensus.cc:2781] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:10:43.941990 18659 raft_consensus.cc:482] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:10:43.942121 18659 raft_consensus.cc:3037] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:43.943575 18659 raft_consensus.cc:504] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:10:43.943851 18659 leader_election.cc:288] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [CANDIDATE]: Term 1 election: Requested vote from peers 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317), 1635c9eb26f14513b0ec873c65662bad (127.2.30.129:44791)
I20260808 10:10:43.944460 18444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1a709b6eab8466fb687a7bfae24419b" candidate_uuid: "b896e6cb7fa74813a48e87cd76098119" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379856a2bbad46dbaee75d1d406a1514"
I20260808 10:10:43.944684 18444 raft_consensus.cc:3037] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:43.944831 18307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1a709b6eab8466fb687a7bfae24419b" candidate_uuid: "b896e6cb7fa74813a48e87cd76098119" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1635c9eb26f14513b0ec873c65662bad"
I20260808 10:10:43.945013 18307 raft_consensus.cc:3037] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:10:43.946113 18444 raft_consensus.cc:2454] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b896e6cb7fa74813a48e87cd76098119 in term 1.
I20260808 10:10:43.946558 18307 raft_consensus.cc:2454] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b896e6cb7fa74813a48e87cd76098119 in term 1.
I20260808 10:10:43.946892 18516 leader_election.cc:302] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 379856a2bbad46dbaee75d1d406a1514, b896e6cb7fa74813a48e87cd76098119; no voters:
I20260808 10:10:43.947078 18659 raft_consensus.cc:2781] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:10:43.947278 18659 raft_consensus.cc:686] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [term 1 LEADER]: Becoming Leader. State: Replica: b896e6cb7fa74813a48e87cd76098119, State: Running, Role: LEADER
I20260808 10:10:43.947674 18659 consensus_queue.cc:237] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:10:43.949579 18156 catalog_manager.cc:5495] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 reported cstate change: term changed from 0 to 1, leader changed from <none> to b896e6cb7fa74813a48e87cd76098119 (127.2.30.131). New cstate: current_term: 1 leader_uuid: "b896e6cb7fa74813a48e87cd76098119" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } health_report { overall_health: UNKNOWN } } }
I20260808 10:10:43.975354 18651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:43.971492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60910 (local address 127.2.30.131:35927)
0808 10:10:43.971625 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 10:10:43.971627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:10:43.971639 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:10:43.971705 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:10:43.971706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:10:43.971739 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:10:43.971826 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:10:43.972090 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.972623 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.973506 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:10:43.974081 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:10:43.974352 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:10:43.974657 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:10:43.974823 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:10:43.974932 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 10:10:43.975152 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 10:10:43.975191 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260808 10:10:43.978751 18307 raft_consensus.cc:1261] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Refusing update from remote peer b896e6cb7fa74813a48e87cd76098119: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:10:43.978739 18444 raft_consensus.cc:1261] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514 [term 1 FOLLOWER]: Refusing update from remote peer b896e6cb7fa74813a48e87cd76098119: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:10:43.979293 18662 consensus_queue.cc:1035] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:10:43.979512 18659 consensus_queue.cc:1035] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [LEADER]: Connected to new peer: Peer: permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:10:43.983263 18667 mvcc.cc:205] Tried to move back new op lower bound from 7316209024931229696 to 7316209024811196416. Current Snapshot: MvccSnapshot[applied={T|T < 7316209024931229696}]
I20260808 10:10:43.987130 18155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:10:43.987295 18155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:10:43.988407 18155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Sending DeleteTablet for 3 replicas of tablet d1a709b6eab8466fb687a7bfae24419b
I20260808 10:10:43.989104 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 18221
I20260808 10:10:43.989161 18562 tablet_service.cc:1514] Processing DeleteTablet for tablet d1a709b6eab8466fb687a7bfae24419b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:10:43 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:60898
I20260808 10:10:43.989188 18286 tablet_service.cc:1514] Processing DeleteTablet for tablet d1a709b6eab8466fb687a7bfae24419b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:10:43 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:41234
I20260808 10:10:43.989789 18423 tablet_service.cc:1514] Processing DeleteTablet for tablet d1a709b6eab8466fb687a7bfae24419b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:10:43 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:41050
I20260808 10:10:43.990096 18677 tablet_replica.cc:330] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514: stopping tablet replica
I20260808 10:10:43.990234 18677 raft_consensus.cc:2229] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:10:43.990352 18677 raft_consensus.cc:2258] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:10:43.990749 18676 tablet_replica.cc:330] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119: stopping tablet replica
I20260808 10:10:43.990875 18676 raft_consensus.cc:2229] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:10:43.990904 18677 ts_tablet_manager.cc:1905] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:10:43.991040 18676 raft_consensus.cc:2258] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:10:43.991578 18676 ts_tablet_manager.cc:1905] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:10:43.993825 18677 ts_tablet_manager.cc:1918] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:10:43.993973 18677 log.cc:1196] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/d1a709b6eab8466fb687a7bfae24419b
I20260808 10:10:43.994216 18676 ts_tablet_manager.cc:1918] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:10:43.994325 18676 log.cc:1196] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/d1a709b6eab8466fb687a7bfae24419b
I20260808 10:10:43.994668 18677 ts_tablet_manager.cc:1939] T d1a709b6eab8466fb687a7bfae24419b P 379856a2bbad46dbaee75d1d406a1514: Deleting consensus metadata
I20260808 10:10:43.994697 18676 ts_tablet_manager.cc:1939] T d1a709b6eab8466fb687a7bfae24419b P b896e6cb7fa74813a48e87cd76098119: Deleting consensus metadata
I20260808 10:10:43.995414 18142 catalog_manager.cc:4841] TS 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317): tablet d1a709b6eab8466fb687a7bfae24419b (table test-table [id=6b1b21d4948b471988518cc0f6680fa1]) successfully deleted
I20260808 10:10:43.995553 18142 catalog_manager.cc:4841] TS b896e6cb7fa74813a48e87cd76098119 (127.2.30.131:35927): tablet d1a709b6eab8466fb687a7bfae24419b (table test-table [id=6b1b21d4948b471988518cc0f6680fa1]) successfully deleted
W20260808 10:10:43.999452 18140 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:44791 (error 108)
I20260808 10:10:43.999655 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 18359
I20260808 10:10:44.006081 18650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:10:44.005847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:44791 (local address 127.0.0.1:41236)
0808 10:10:44.005935 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 10:10:44.006016 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:44791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 10:10:44.006294 18140 catalog_manager.cc:4584] TS 1635c9eb26f14513b0ec873c65662bad (127.2.30.129:44791): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d1a709b6eab8466fb687a7bfae24419b: Network error: Client connection negotiation failed: client connection to 127.2.30.129:44791: connect: Connection refused (error 111)
I20260808 10:10:44.010941 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 18498
I20260808 10:10:44.017781 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 18123
I20260808 10:11:04.026544 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38189
--webserver_interface=127.2.30.190
--webserver_port=34793
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:39861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:38189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:04.122138 18679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:04.122653 18679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:04.122874 18679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:04.124770 18679 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 10:11:04.124856 18679 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 10:11:04.124883 18679 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 10:11:04.124915 18679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:11:04.124954 18679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:04.124984 18679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:04.125020 18679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:11:04.125046 18679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:11:04.126935 18679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:38189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=34793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:04.127329 18679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:04.127633 18679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:04.139493 18686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:04.139521 18688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:04.139495 18679 server_base.cc:1034] running on GCE node
W20260808 10:11:04.139519 18685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:04.140090 18679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:04.140527 18679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:04.141719 18679 hybrid_clock.cc:648] HybridClock initialized: now 1786183864141712 us; error 49 us; skew 500 ppm
Aug 08 10:11:04 dist-test-slave-w206 krb5kdc[18110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:04.144954 18679 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:11:04.145619 18679 webserver.cc:460] Webserver started at http://127.2.30.190:34793/ using document root <none> and password file <none>
I20260808 10:11:04.145933 18679 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:04.146039 18679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:04.147877 18679 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:11:04.148841 18696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:04.149133 18679 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:04.149269 18679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "7102975c4a3946a5a00f9b9837293022"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "86d2e5f26b9f05b2afa08e167e9d3aec"
server_key_iv: "6f86eb2343a28d0bfd37688789d36c79"
server_key_version: "encryptionkey@0"
I20260808 10:11:04.149835 18679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:04.160885 18679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:04.163745 18679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:04.163955 18679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:04.169591 18679 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:38189
I20260808 10:11:04.169631 18753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:38189 every 8 connection(s)
I20260808 10:11:04.170187 18679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:11:04.172180 18755 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:11:04.173151 18755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Bootstrap starting.
I20260808 10:11:04.174429 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 18679
I20260808 10:11:04.174798 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:44791
--local_ip_for_outbound_sockets=127.2.30.129
--tserver_master_addrs=127.2.30.190:38189
--webserver_port=34293
--webserver_interface=127.2.30.129
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:39861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:11:04.185896 18755 log.cc:824] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:04.189636 18755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:11:04.190122 18755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Bootstrap complete.
I20260808 10:11:04.194319 18755 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } }
I20260808 10:11:04.194736 18755 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7102975c4a3946a5a00f9b9837293022, State: Initialized, Role: FOLLOWER
I20260808 10:11:04.194932 18755 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } }
I20260808 10:11:04.195045 18755 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:11:04.195106 18755 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:11:04.195155 18755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 1 FOLLOWER]: Advancing to term 2
I20260808 10:11:04.196760 18755 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } }
I20260808 10:11:04.196895 18755 leader_election.cc:302] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7102975c4a3946a5a00f9b9837293022; no voters:
I20260808 10:11:04.197088 18755 leader_election.cc:288] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 10:11:04.197218 18759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 10:11:04.197389 18755 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:11:04.197448 18759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 2 LEADER]: Becoming Leader. State: Replica: 7102975c4a3946a5a00f9b9837293022, State: Running, Role: LEADER
I20260808 10:11:04.197566 18759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } }
I20260808 10:11:04.197897 18760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7102975c4a3946a5a00f9b9837293022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } } }
I20260808 10:11:04.197984 18760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:04.198237 18761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7102975c4a3946a5a00f9b9837293022. Latest consensus state: current_term: 2 leader_uuid: "7102975c4a3946a5a00f9b9837293022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } } }
I20260808 10:11:04.198383 18761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:04.198580 18768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:11:04.199608 18768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:11:04.199740 18768 catalog_manager.cc:1259] Loaded cluster ID: 3999e8f231ed49bd8d5f6647b2b0f3fa
I20260808 10:11:04.199776 18768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:11:04.200704 18768 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:11:04.200923 18768 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Loaded TSK: 0
I20260808 10:11:04.210835 18768 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Generated new TSK 1
I20260808 10:11:04.211014 18768 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:11:04.304811 18757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:04.305119 18757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:04.305168 18757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:04.306667 18757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:04.306746 18757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:04.306804 18757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:11:04.309478 18757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:44791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=34293
--tserver_master_addrs=127.2.30.190:38189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:04.310036 18757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:04.310379 18757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:04.317440 18783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:04.317600 18786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:04.317734 18784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:04.319500 18757 server_base.cc:1034] running on GCE node
I20260808 10:11:04.320065 18757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:04.320477 18757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:04.321781 18757 hybrid_clock.cc:648] HybridClock initialized: now 1786183864321697 us; error 93 us; skew 500 ppm
Aug 08 10:11:04 dist-test-slave-w206 krb5kdc[18110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:04.326081 18757 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:11:04.326869 18757 webserver.cc:460] Webserver started at http://127.2.30.129:34293/ using document root <none> and password file <none>
I20260808 10:11:04.327256 18757 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:04.327361 18757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:04.329705 18757 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:11:04.330927 18793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:04.331487 18757 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:04.331622 18757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "1635c9eb26f14513b0ec873c65662bad"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "d751ffd5da5b33481a9b2f7db24a73fb"
server_key_iv: "63621ec6b71fc1e7781dffd591b6d0ac"
server_key_version: "encryptionkey@0"
I20260808 10:11:04.332193 18757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:04.344175 18757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:04.348217 18757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:04.348418 18757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:04.348722 18757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:04.349306 18800 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:11:04.353237 18757 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:11:04.353405 18757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260808 10:11:04.353466 18757 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:11:04.354795 18757 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:11:04.354902 18757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:04.355077 18800 tablet_bootstrap.cc:492] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: Bootstrap starting.
I20260808 10:11:04.366684 18757 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:44791
I20260808 10:11:04.366945 18910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:44791 every 8 connection(s)
I20260808 10:11:04.367309 18757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:11:04.367908 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 18757
I20260808 10:11:04.368244 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:36317
--local_ip_for_outbound_sockets=127.2.30.130
--tserver_master_addrs=127.2.30.190:38189
--webserver_port=33171
--webserver_interface=127.2.30.130
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:39861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:11:04.372586 18800 log.cc:824] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:04.375738 18800 tablet_bootstrap.cc:492] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:11:04.376138 18800 tablet_bootstrap.cc:492] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: Bootstrap complete.
I20260808 10:11:04.377632 18800 ts_tablet_manager.cc:1397] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: Time spent bootstrapping tablet: real 0.023s user 0.010s sys 0.009s
I20260808 10:11:04.379420 18800 raft_consensus.cc:348] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:04.379822 18800 raft_consensus.cc:729] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1635c9eb26f14513b0ec873c65662bad, State: Initialized, Role: FOLLOWER
I20260808 10:11:04.380040 18800 consensus_queue.cc:260] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:04.380411 18800 ts_tablet_manager.cc:1428] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
Aug 08 10:11:04 dist-test-slave-w206 krb5kdc[18110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:11:04.390225 18914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.370931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:36317)
0808 10:11:04.371935 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:04.371992 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:04.372935 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:04.373852 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:04.373857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:04.374306 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:04.374625 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:04.374634 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.376858 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.376860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:04.378670 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:04.378676 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.380052 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.380054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:04.380329 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.381123 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:11:04.381135 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:04.383343 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:04.386379 (+ 3036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:04.386395 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:04.386399 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:04.386819 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:04.387434 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:04.387437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:04.387438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:04.387849 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:04.388506 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:04.388515 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:04.389145 (+ 630us) client_negotiation.cc:769] Sending connection context
0808 10:11:04.389668 (+ 523us) client_negotiation.cc:240] Negotiation successful
0808 10:11:04.389943 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"thread_start_us":170,"threads_started":1}
I20260808 10:11:04.390242 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.371123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:36317 (local address 127.2.30.190:38189)
0808 10:11:04.373182 (+ 2059us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:04.373192 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:04.373252 (+ 60us) server_negotiation.cc:407] Connection header received
0808 10:11:04.373408 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:04.373416 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:04.373492 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:04.373665 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:04.374795 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.376674 (+ 1879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.378840 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.379652 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.379786 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.383507 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:04.383513 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:04.383514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:04.383531 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:04.386210 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:04.386967 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:04.386993 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:04.386993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:04.387161 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:04.387980 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:04.387982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:04.387982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:04.388349 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:04.388443 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:04.389815 (+ 1372us) server_negotiation.cc:299] Negotiation successful
0808 10:11:04.390004 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1991,"thread_start_us":109,"threads_started":1}
I20260808 10:11:04.391767 18911 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38189
I20260808 10:11:04.391908 18911 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:04.392138 18911 heartbeater.cc:511] Master 127.2.30.190:38189 requested a full tablet report, sending...
I20260808 10:11:04.393069 18714 ts_manager.cc:194] Registered new tserver with Master: 1635c9eb26f14513b0ec873c65662bad (127.2.30.129:44791)
I20260808 10:11:04.394369 18714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:36317
I20260808 10:11:04.395545 18911 heartbeater.cc:503] Master 127.2.30.190:38189 was elected leader, sending a full tablet report...
Aug 08 10:11:04 dist-test-slave-w206 krb5kdc[18110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for kudu/127.2.30.129@KRBTEST.COM
I20260808 10:11:04.407933 18918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.393824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:44791 (local address 127.0.0.1:40224)
0808 10:11:04.394432 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:04.394674 (+ 242us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:04.394767 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:04.395077 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:04.395083 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:04.395474 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:04.395583 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:04.395585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.397069 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.397071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:04.399323 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:04.399331 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.400166 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.400167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:04.400269 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.400943 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:11:04.400949 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:04.403129 (+ 2180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:04.405662 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:04.405674 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:04.405676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:04.406152 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:04.406673 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:04.406674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:04.406675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:04.406781 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:04.407477 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:04.407489 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:04.407556 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 10:11:04.407626 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 10:11:04.407807 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":132,"threads_started":1}
I20260808 10:11:04.408776 18919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.394142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40224 (local address 127.2.30.129:44791)
0808 10:11:04.394610 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:04.394615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:04.394849 (+ 234us) server_negotiation.cc:407] Connection header received
0808 10:11:04.394895 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:04.394897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:04.394916 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:04.395010 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:04.395777 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.396959 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.399487 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.399996 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.400082 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.403296 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:04.403317 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:04.403317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:04.403330 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:04.405515 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:04.406281 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:04.406285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:04.406286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:04.406468 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:04.407083 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:04.407085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:04.407085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:04.407354 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:04.407443 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:04.408498 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0808 10:11:04.408663 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":179,"threads_started":1}
I20260808 10:11:04.410060 18842 tablet_service.cc:1514] Processing DeleteTablet for tablet d1a709b6eab8466fb687a7bfae24419b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:10:43 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:40224
I20260808 10:11:04.411049 18920 tablet_replica.cc:330] stopping tablet replica
I20260808 10:11:04.411185 18920 raft_consensus.cc:2229] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:11:04.411270 18920 raft_consensus.cc:2258] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:04.411737 18920 ts_tablet_manager.cc:1905] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:11:04.415386 18920 ts_tablet_manager.cc:1918] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:11:04.415511 18920 log.cc:1196] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal/wals/d1a709b6eab8466fb687a7bfae24419b
I20260808 10:11:04.415787 18920 ts_tablet_manager.cc:1939] T d1a709b6eab8466fb687a7bfae24419b P 1635c9eb26f14513b0ec873c65662bad: Deleting consensus metadata
I20260808 10:11:04.416760 18698 catalog_manager.cc:4841] TS 1635c9eb26f14513b0ec873c65662bad (127.2.30.129:44791): tablet d1a709b6eab8466fb687a7bfae24419b (table test-table [id=6b1b21d4948b471988518cc0f6680fa1]) successfully deleted
W20260808 10:11:04.480950 18913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:04.481199 18913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:04.481326 18913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:04.482625 18913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:04.482683 18913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:04.482719 18913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:11:04.484390 18913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:36317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=33171
--tserver_master_addrs=127.2.30.190:38189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:04.484664 18913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:04.484892 18913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:04.490775 18927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:04.490875 18913 server_base.cc:1034] running on GCE node
W20260808 10:11:04.490847 18926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:04.490835 18929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:04.491377 18913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:04.491676 18913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:04.492874 18913 hybrid_clock.cc:648] HybridClock initialized: now 1786183864492848 us; error 38 us; skew 500 ppm
Aug 08 10:11:04 dist-test-slave-w206 krb5kdc[18110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:04.496008 18913 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:11:04.496603 18913 webserver.cc:460] Webserver started at http://127.2.30.130:33171/ using document root <none> and password file <none>
I20260808 10:11:04.496940 18913 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:04.497107 18913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:04.499083 18913 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:04.500088 18936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:04.500560 18913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:04.500818 18913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "379856a2bbad46dbaee75d1d406a1514"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "259c2784db517bc7bb66994448a68394"
server_key_iv: "801b17b01b979473fd1ea23c90182e28"
server_key_version: "encryptionkey@0"
I20260808 10:11:04.501596 18913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:04.514863 18913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:04.520176 18913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:04.520423 18913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:04.520749 18913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:04.521195 18913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:04.521265 18913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:04.521320 18913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:04.521371 18913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:04.529779 18913 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:36317
I20260808 10:11:04.529955 19050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:36317 every 8 connection(s)
I20260808 10:11:04.530515 18913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:11:04.537462 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 18913
I20260808 10:11:04.537905 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:35927
--local_ip_for_outbound_sockets=127.2.30.131
--tserver_master_addrs=127.2.30.190:38189
--webserver_port=37433
--webserver_interface=127.2.30.131
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:39861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 10:11:04 dist-test-slave-w206 krb5kdc[18110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:11:04.544728 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.531659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:42055 (local address 127.2.30.190:38189)
0808 10:11:04.531712 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:04.531714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:04.531794 (+ 80us) server_negotiation.cc:407] Connection header received
0808 10:11:04.532506 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:04.532508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:04.532535 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:04.532675 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:04.534044 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.535235 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.536612 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.536888 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.537013 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.540212 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:04.540215 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:04.540215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:04.540227 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:04.542591 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:04.543285 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:04.543286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:04.543286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:04.543365 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:04.543707 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:04.543709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:04.543709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:04.543901 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:04.543983 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:04.544443 (+ 460us) server_negotiation.cc:299] Negotiation successful
0808 10:11:04.544574 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:11:04.544862 19053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.531444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:42055)
0808 10:11:04.531650 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:04.531704 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:04.532288 (+ 584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:04.532812 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:04.532817 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:04.533559 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:04.533829 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:04.533834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.535384 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.535386 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:04.536477 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:04.536481 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.537038 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.537039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:04.537246 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.538154 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:11:04.538171 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:04.540057 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:04.542788 (+ 2731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:04.542803 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:04.542807 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:04.543191 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:04.543488 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:04.543489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:04.543489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:04.543621 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:04.544056 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:04.544067 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:04.544272 (+ 205us) client_negotiation.cc:769] Sending connection context
0808 10:11:04.544388 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 10:11:04.544563 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":121,"threads_started":1}
I20260808 10:11:04.545892 19051 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38189
I20260808 10:11:04.546023 19051 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:04.546301 19051 heartbeater.cc:511] Master 127.2.30.190:38189 requested a full tablet report, sending...
I20260808 10:11:04.546947 18714 ts_manager.cc:194] Registered new tserver with Master: 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317)
I20260808 10:11:04.547595 18714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:42055
W20260808 10:11:04.645076 19054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:04.645358 19054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:04.645390 19054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:04.647229 19054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:04.647312 19054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:04.647347 19054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:11:04.650228 19054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:35927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=37433
--tserver_master_addrs=127.2.30.190:38189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:04.650586 19054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:04.650813 19054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:11:04.657094 19054 server_base.cc:1034] running on GCE node
W20260808 10:11:04.657006 19059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:04.657014 19062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:04.657006 19060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:04.657601 19054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:04.657950 19054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:04.659240 19054 hybrid_clock.cc:648] HybridClock initialized: now 1786183864659225 us; error 46 us; skew 500 ppm
Aug 08 10:11:04 dist-test-slave-w206 krb5kdc[18110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:04.662534 19054 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:11:04.663157 19054 webserver.cc:460] Webserver started at http://127.2.30.131:37433/ using document root <none> and password file <none>
I20260808 10:11:04.663407 19054 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:04.663475 19054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:04.665009 19054 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:11:04.666131 19070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:04.666644 19054 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:04.666883 19054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "b896e6cb7fa74813a48e87cd76098119"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "fb98b0dc6592c56fee9441b659c543d1"
server_key_iv: "53b4d90ed7cc6f94d830e71c7c6edc8c"
server_key_version: "encryptionkey@0"
I20260808 10:11:04.667500 19054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:04.692752 19054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:04.695950 19054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:04.696091 19054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:04.696314 19054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:04.696813 19054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:04.696902 19054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:04.696936 19054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:04.696950 19054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:04.705209 19054 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:35927
I20260808 10:11:04.705410 19185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:35927 every 8 connection(s)
I20260808 10:11:04.705850 19054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:11:04.708096 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 19054
Aug 08 10:11:04 dist-test-slave-w206 krb5kdc[18110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183864, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:11:04.718935 19190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.708786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51708 (local address 127.2.30.190:38189)
0808 10:11:04.708994 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:04.708996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:04.709611 (+ 615us) server_negotiation.cc:407] Connection header received
0808 10:11:04.709708 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:04.709709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:04.709724 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:04.709794 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:04.710489 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.711194 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.712164 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.712400 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.712481 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.714275 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:04.714277 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:04.714278 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:04.714286 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:04.716677 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:04.717350 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:04.717353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:04.717353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:04.717526 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:04.718085 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:04.718087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:04.718088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:04.718305 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:04.718548 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:04.718660 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 10:11:04.718782 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":174,"threads_started":1}
I20260808 10:11:04.720067 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.706906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:60371 (local address 127.2.30.190:38189)
0808 10:11:04.707093 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:04.707097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:04.707228 (+ 131us) server_negotiation.cc:407] Connection header received
0808 10:11:04.707815 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:04.707816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:04.707828 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:04.707898 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:04.709523 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.710087 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.711838 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.712071 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.712190 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.715450 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:04.715451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:04.715452 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:04.715459 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:04.717515 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:04.718082 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:04.718084 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:04.718084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:04.718181 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:04.718501 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:04.718501 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:04.718502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:04.718794 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:04.719291 (+ 497us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:04.719820 (+ 529us) server_negotiation.cc:299] Negotiation successful
0808 10:11:04.719962 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260808 10:11:04.720336 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.706850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:60371)
0808 10:11:04.707096 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:04.707144 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:04.707656 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:04.708087 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:04.708093 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:04.709113 (+ 1020us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:04.709350 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:04.709357 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.710368 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.710370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:04.711682 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:04.711687 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.712204 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.712205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:04.712604 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.713471 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:11:04.713481 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:04.715255 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:04.717629 (+ 2374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:04.717650 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:04.717658 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:04.717968 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:04.718276 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:04.718277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:04.718277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:04.718412 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:04.719378 (+ 966us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:04.719388 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:04.719572 (+ 184us) client_negotiation.cc:769] Sending connection context
0808 10:11:04.719771 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 10:11:04.720022 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":98,"threads_started":1}
I20260808 10:11:04.721371 19186 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38189
I20260808 10:11:04.721791 19186 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:04.722225 19186 heartbeater.cc:511] Master 127.2.30.190:38189 requested a full tablet report, sending...
I20260808 10:11:04.723090 18714 ts_manager.cc:194] Registered new tserver with Master: b896e6cb7fa74813a48e87cd76098119 (127.2.30.131:35927)
I20260808 10:11:04.723614 18714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:60371
I20260808 10:11:04.731235 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:11:04.735162 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.732206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51710 (local address 127.2.30.190:38189)
0808 10:11:04.732361 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:04.732363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:04.732392 (+ 29us) server_negotiation.cc:407] Connection header received
0808 10:11:04.732432 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:04.732432 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:04.732474 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:04.732533 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:11:04.732722 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.733304 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.734200 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.734403 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.734478 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.734767 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:11:04.734845 (+ 78us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 10:11:04.735091 (+ 246us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51710: token expired
Metrics: {"server-negotiator.queue_time_us":131}
W20260808 10:11:04.735330 18915 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51710: token expired
W20260808 10:11:04.735558 18637 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.30.190:38189)
I20260808 10:11:04.743209 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.735863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51722 (local address 127.2.30.190:38189)
0808 10:11:04.735926 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:04.735927 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:04.735938 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:11:04.735968 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:04.735969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:04.735980 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:04.736048 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:04.736656 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.737180 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.737976 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.738190 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.738271 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.739888 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:04.739889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:04.739889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:04.739915 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:04.741806 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:04.742242 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:04.742243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:04.742243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:04.742308 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:04.742540 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:04.742541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:04.742541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:04.742679 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:04.742807 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:04.742936 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 10:11:04.743065 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:11:04.745232 18714 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:11:04.747957 18714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:11:04.751750 18842 tablet_service.cc:1467] Processing CreateTablet for tablet fdb8a12aeadd4249a9a825f9e65d22fe (DEFAULT_TABLE table=test-table [id=2613f8fc54a44b8abadca83940631ab7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:04.752067 18842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdb8a12aeadd4249a9a825f9e65d22fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:04.753844 18800 tablet_bootstrap.cc:492] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad: Bootstrap starting.
I20260808 10:11:04.754221 18800 tablet_bootstrap.cc:654] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:04.755158 18800 tablet_bootstrap.cc:492] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad: No bootstrap required, opened a new log
I20260808 10:11:04.755223 18800 ts_tablet_manager.cc:1397] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:04.755605 18800 raft_consensus.cc:348] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:04.755719 18800 raft_consensus.cc:374] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:04.755784 18800 raft_consensus.cc:729] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1635c9eb26f14513b0ec873c65662bad, State: Initialized, Role: FOLLOWER
I20260808 10:11:04.755837 18800 consensus_queue.cc:260] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:04.755951 18800 ts_tablet_manager.cc:1428] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:04.756808 18918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.751484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:35927 (local address 127.0.0.1:44494)
0808 10:11:04.751547 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:04.751562 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:04.751609 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:04.752153 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:04.752154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:04.752157 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:04.752337 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:04.752342 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.754434 (+ 2092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.754435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:04.755505 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:04.755508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.756507 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.756508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:04.756597 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.756602 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:11:04.756661 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 10:11:04.756720 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":1}
I20260808 10:11:04.756883 19197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.751614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44494 (local address 127.2.30.131:35927)
0808 10:11:04.751909 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:04.751933 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:04.751945 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:11:04.751998 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:04.752000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:04.752017 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:04.752146 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:04.752546 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.753727 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.755605 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.756391 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.756477 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.756538 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:04.756684 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 10:11:04.756732 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":170,"threads_started":1}
I20260808 10:11:04.757122 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.751588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:36317 (local address 127.0.0.1:44092)
0808 10:11:04.751715 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:04.751724 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:04.751768 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:04.752362 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:04.752363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:04.752366 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:04.752414 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:04.752415 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.753556 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.753558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:04.755041 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:04.755044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.756485 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.756486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:04.756559 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.756580 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 10:11:04.756634 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 10:11:04.756686 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
I20260808 10:11:04.757366 19198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.751943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44092 (local address 127.2.30.130:36317)
0808 10:11:04.752155 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:04.752159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:04.752170 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:11:04.752212 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:04.752215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:04.752230 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:04.752321 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:04.752508 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.753464 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.755152 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.756139 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.756214 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.756283 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:04.756776 (+ 493us) server_negotiation.cc:299] Negotiation successful
0808 10:11:04.756829 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":96,"threads_started":1}
I20260808 10:11:04.757644 19118 tablet_service.cc:1467] Processing CreateTablet for tablet fdb8a12aeadd4249a9a825f9e65d22fe (DEFAULT_TABLE table=test-table [id=2613f8fc54a44b8abadca83940631ab7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:04.758065 19118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdb8a12aeadd4249a9a825f9e65d22fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:04.758157 18984 tablet_service.cc:1467] Processing CreateTablet for tablet fdb8a12aeadd4249a9a825f9e65d22fe (DEFAULT_TABLE table=test-table [id=2613f8fc54a44b8abadca83940631ab7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:04.758555 18984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdb8a12aeadd4249a9a825f9e65d22fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:04.762032 19200 tablet_bootstrap.cc:492] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514: Bootstrap starting.
I20260808 10:11:04.762751 19200 tablet_bootstrap.cc:654] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:04.763093 19201 tablet_bootstrap.cc:492] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119: Bootstrap starting.
I20260808 10:11:04.763595 19200 log.cc:824] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:04.764268 19201 tablet_bootstrap.cc:654] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:04.765292 19201 log.cc:824] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:04.765341 19200 tablet_bootstrap.cc:492] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514: No bootstrap required, opened a new log
I20260808 10:11:04.765441 19200 ts_tablet_manager.cc:1397] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 10:11:04.766755 19201 tablet_bootstrap.cc:492] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119: No bootstrap required, opened a new log
I20260808 10:11:04.766860 19201 ts_tablet_manager.cc:1397] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 10:11:04.767891 19200 raft_consensus.cc:348] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:04.768126 19200 raft_consensus.cc:374] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:04.768168 19200 raft_consensus.cc:729] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 379856a2bbad46dbaee75d1d406a1514, State: Initialized, Role: FOLLOWER
I20260808 10:11:04.768301 19200 consensus_queue.cc:260] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:04.768666 19200 ts_tablet_manager.cc:1428] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260808 10:11:04.768711 19051 heartbeater.cc:503] Master 127.2.30.190:38189 was elected leader, sending a full tablet report...
I20260808 10:11:04.769138 19201 raft_consensus.cc:348] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:04.769399 19201 raft_consensus.cc:374] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:04.769444 19201 raft_consensus.cc:729] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b896e6cb7fa74813a48e87cd76098119, State: Initialized, Role: FOLLOWER
I20260808 10:11:04.769627 19201 consensus_queue.cc:260] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:04.770013 19201 ts_tablet_manager.cc:1428] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 10:11:04.770035 19186 heartbeater.cc:503] Master 127.2.30.190:38189 was elected leader, sending a full tablet report...
W20260808 10:11:04.781638 19052 tablet.cc:2369] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:11:04.872354 18912 tablet.cc:2369] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:11:04.956588 19187 tablet.cc:2369] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:11:04.966790 19205 raft_consensus.cc:482] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:11:04.967141 19205 raft_consensus.cc:504] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:04.967752 19205 leader_election.cc:288] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317), 1635c9eb26f14513b0ec873c65662bad (127.2.30.129:44791)
I20260808 10:11:04.972963 19188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.968290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:44791 (local address 127.2.30.131:32949)
0808 10:11:04.968392 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:04.968411 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:04.968492 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:04.968931 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:04.968932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:04.968936 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:04.969032 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:04.969035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.969926 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.969929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:04.971697 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:04.971704 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.972544 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.972546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:04.972640 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.972645 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:11:04.972797 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 10:11:04.972858 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":1}
I20260808 10:11:04.973052 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.968359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:32949 (local address 127.2.30.129:44791)
0808 10:11:04.968589 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:04.968593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:04.968610 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:11:04.968687 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:04.968688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:04.968730 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:04.968834 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:04.969141 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.969779 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.971871 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.972417 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.972499 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.972563 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:04.972851 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 10:11:04.972916 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":119,"threads_started":1}
I20260808 10:11:04.973143 19206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.968323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:36317 (local address 127.2.30.131:47385)
0808 10:11:04.968592 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:04.968606 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:04.968739 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:04.969068 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:04.969069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:04.969101 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:04.969186 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:04.969188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.970587 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.970590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:04.971939 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:04.971943 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.972862 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.972863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:04.972938 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.972942 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:11:04.973008 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:11:04.973052 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":139,"threads_started":1}
I20260808 10:11:04.973176 19198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.968424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:47385 (local address 127.2.30.130:36317)
0808 10:11:04.968589 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:04.968592 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:04.968736 (+ 144us) server_negotiation.cc:407] Connection header received
0808 10:11:04.968932 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:04.968934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:04.968952 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:04.969058 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:04.969455 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.970406 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.972085 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.972731 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.972829 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.972840 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:04.973040 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 10:11:04.973096 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260808 10:11:04.973440 18862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb8a12aeadd4249a9a825f9e65d22fe" candidate_uuid: "b896e6cb7fa74813a48e87cd76098119" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1635c9eb26f14513b0ec873c65662bad" is_pre_election: true
I20260808 10:11:04.973489 19004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb8a12aeadd4249a9a825f9e65d22fe" candidate_uuid: "b896e6cb7fa74813a48e87cd76098119" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379856a2bbad46dbaee75d1d406a1514" is_pre_election: true
I20260808 10:11:04.973686 19004 raft_consensus.cc:2454] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b896e6cb7fa74813a48e87cd76098119 in term 0.
I20260808 10:11:04.973673 18862 raft_consensus.cc:2454] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b896e6cb7fa74813a48e87cd76098119 in term 0.
I20260808 10:11:04.973985 19073 leader_election.cc:302] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 379856a2bbad46dbaee75d1d406a1514, b896e6cb7fa74813a48e87cd76098119; no voters:
I20260808 10:11:04.974181 19205 raft_consensus.cc:2781] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:11:04.974249 19205 raft_consensus.cc:482] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:11:04.974273 19205 raft_consensus.cc:3037] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:04.975226 19205 raft_consensus.cc:504] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:04.975467 19205 leader_election.cc:288] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [CANDIDATE]: Term 1 election: Requested vote from peers 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317), 1635c9eb26f14513b0ec873c65662bad (127.2.30.129:44791)
I20260808 10:11:04.975656 19004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb8a12aeadd4249a9a825f9e65d22fe" candidate_uuid: "b896e6cb7fa74813a48e87cd76098119" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379856a2bbad46dbaee75d1d406a1514"
I20260808 10:11:04.975714 18862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb8a12aeadd4249a9a825f9e65d22fe" candidate_uuid: "b896e6cb7fa74813a48e87cd76098119" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1635c9eb26f14513b0ec873c65662bad"
I20260808 10:11:04.975791 19004 raft_consensus.cc:3037] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:04.975991 18862 raft_consensus.cc:3037] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:04.976681 19004 raft_consensus.cc:2454] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b896e6cb7fa74813a48e87cd76098119 in term 1.
I20260808 10:11:04.976915 19073 leader_election.cc:302] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 379856a2bbad46dbaee75d1d406a1514, b896e6cb7fa74813a48e87cd76098119; no voters:
I20260808 10:11:04.976997 18862 raft_consensus.cc:2454] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b896e6cb7fa74813a48e87cd76098119 in term 1.
I20260808 10:11:04.977044 19205 raft_consensus.cc:2781] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:04.977219 19205 raft_consensus.cc:686] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [term 1 LEADER]: Becoming Leader. State: Replica: b896e6cb7fa74813a48e87cd76098119, State: Running, Role: LEADER
I20260808 10:11:04.977363 19205 consensus_queue.cc:237] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:04.978760 18713 catalog_manager.cc:5495] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 reported cstate change: term changed from 0 to 1, leader changed from <none> to b896e6cb7fa74813a48e87cd76098119 (127.2.30.131). New cstate: current_term: 1 leader_uuid: "b896e6cb7fa74813a48e87cd76098119" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } health_report { overall_health: UNKNOWN } } }
I20260808 10:11:04.988255 19197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:04.985388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44502 (local address 127.2.30.131:35927)
0808 10:11:04.985476 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:04.985478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:04.985484 (+ 6us) server_negotiation.cc:407] Connection header received
0808 10:11:04.985601 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:04.985603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:04.985630 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:04.985781 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:11:04.986069 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.986560 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.987385 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:04.987585 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:04.987667 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:04.987857 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:11:04.987972 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:11:04.988075 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:04.988100 (+ 25us) server_negotiation.cc:299] Negotiation successful
0808 10:11:04.988123 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:11:04.990473 18862 raft_consensus.cc:1261] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Refusing update from remote peer b896e6cb7fa74813a48e87cd76098119: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:11:04.990814 19004 raft_consensus.cc:1261] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514 [term 1 FOLLOWER]: Refusing update from remote peer b896e6cb7fa74813a48e87cd76098119: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:11:04.991322 19208 consensus_queue.cc:1035] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [LEADER]: Connected to new peer: Peer: permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:11:04.991503 19205 consensus_queue.cc:1035] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:11:04.995345 19211 mvcc.cc:205] Tried to move back new op lower bound from 7316209110997512192 to 7316209110948102144. Current Snapshot: MvccSnapshot[applied={T|T < 7316209110997512192}]
I20260808 10:11:04.998410 18714 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:11:04.998526 18714 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:11:04.999446 18714 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Sending DeleteTablet for 3 replicas of tablet fdb8a12aeadd4249a9a825f9e65d22fe
I20260808 10:11:04.999802 19118 tablet_service.cc:1514] Processing DeleteTablet for tablet fdb8a12aeadd4249a9a825f9e65d22fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:04 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:44494
I20260808 10:11:04.999840 18842 tablet_service.cc:1514] Processing DeleteTablet for tablet fdb8a12aeadd4249a9a825f9e65d22fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:04 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:40224
I20260808 10:11:04.999879 18984 tablet_service.cc:1514] Processing DeleteTablet for tablet fdb8a12aeadd4249a9a825f9e65d22fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:04 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:44092
I20260808 10:11:05.000150 19220 tablet_replica.cc:330] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119: stopping tablet replica
I20260808 10:11:05.000243 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 18757
I20260808 10:11:05.000272 19220 raft_consensus.cc:2229] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:11:05.000730 19220 raft_consensus.cc:2258] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:05.002501 19220 ts_tablet_manager.cc:1905] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:11:05.005473 19220 ts_tablet_manager.cc:1918] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:11:05.005641 19220 log.cc:1196] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/fdb8a12aeadd4249a9a825f9e65d22fe
I20260808 10:11:05.006127 19220 ts_tablet_manager.cc:1939] T fdb8a12aeadd4249a9a825f9e65d22fe P b896e6cb7fa74813a48e87cd76098119: Deleting consensus metadata
I20260808 10:11:05.006790 18699 catalog_manager.cc:4841] TS b896e6cb7fa74813a48e87cd76098119 (127.2.30.131:35927): tablet fdb8a12aeadd4249a9a825f9e65d22fe (table test-table [id=2613f8fc54a44b8abadca83940631ab7]) successfully deleted
I20260808 10:11:05.007136 19222 tablet_replica.cc:330] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514: stopping tablet replica
I20260808 10:11:05.007404 19222 raft_consensus.cc:2229] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:11:05.007580 19222 raft_consensus.cc:2258] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:05.008098 19222 ts_tablet_manager.cc:1905] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:11:05.010061 19222 ts_tablet_manager.cc:1918] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:11:05.010192 19222 log.cc:1196] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/fdb8a12aeadd4249a9a825f9e65d22fe
I20260808 10:11:05.010685 19222 ts_tablet_manager.cc:1939] T fdb8a12aeadd4249a9a825f9e65d22fe P 379856a2bbad46dbaee75d1d406a1514: Deleting consensus metadata
W20260808 10:11:05.011178 18698 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:44791 (error 108)
I20260808 10:11:05.011317 18699 catalog_manager.cc:4841] TS 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317): tablet fdb8a12aeadd4249a9a825f9e65d22fe (table test-table [id=2613f8fc54a44b8abadca83940631ab7]) successfully deleted
I20260808 10:11:05.012022 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 18913
I20260808 10:11:05.012172 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:05.012013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:44791 (local address 127.0.0.1:40236)
0808 10:11:05.012078 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:05.012126 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:44791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 10:11:05.013298 18698 catalog_manager.cc:4584] TS 1635c9eb26f14513b0ec873c65662bad (127.2.30.129:44791): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fdb8a12aeadd4249a9a825f9e65d22fe: Network error: Client connection negotiation failed: client connection to 127.2.30.129:44791: connect: Connection refused (error 111)
I20260808 10:11:05.022325 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 19054
I20260808 10:11:05.030301 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 18679
I20260808 10:11:25.039675 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38189
--webserver_interface=127.2.30.190
--webserver_port=34793
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:39861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:38189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:25.139616 19224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:25.139886 19224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:25.139936 19224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:25.141330 19224 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 10:11:25.141386 19224 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 10:11:25.141398 19224 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 10:11:25.141412 19224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:11:25.141425 19224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:25.141439 19224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:25.141451 19224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:11:25.141464 19224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:11:25.143297 19224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:38189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=34793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:25.143592 19224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:25.143823 19224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:11:25.151451 19224 server_base.cc:1034] running on GCE node
W20260808 10:11:25.151379 19233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:25.151407 19230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:25.151408 19229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:25.152073 19224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:25.152594 19224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:25.153818 19224 hybrid_clock.cc:648] HybridClock initialized: now 1786183885153834 us; error 68 us; skew 500 ppm
Aug 08 10:11:25 dist-test-slave-w206 krb5kdc[18110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183885, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:25.157486 19224 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:11:25.158104 19224 webserver.cc:460] Webserver started at http://127.2.30.190:34793/ using document root <none> and password file <none>
I20260808 10:11:25.158387 19224 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:25.158444 19224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:25.160193 19224 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:25.161234 19240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:25.161537 19224 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:25.161676 19224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "7102975c4a3946a5a00f9b9837293022"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "86d2e5f26b9f05b2afa08e167e9d3aec"
server_key_iv: "6f86eb2343a28d0bfd37688789d36c79"
server_key_version: "encryptionkey@0"
I20260808 10:11:25.162143 19224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:25.181352 19224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:25.184592 19224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:25.184938 19224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:25.190938 19224 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:38189
I20260808 10:11:25.191131 19295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:38189 every 8 connection(s)
I20260808 10:11:25.191687 19224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:11:25.193533 19296 sys_catalog.cc:263] Verifying existing consensus state
I20260808 10:11:25.194280 19296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Bootstrap starting.
I20260808 10:11:25.198534 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 19224
I20260808 10:11:25.199195 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:44791
--local_ip_for_outbound_sockets=127.2.30.129
--tserver_master_addrs=127.2.30.190:38189
--webserver_port=34293
--webserver_interface=127.2.30.129
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:39861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:11:25.206506 19296 log.cc:824] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:25.209277 19296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:11:25.209944 19296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Bootstrap complete.
I20260808 10:11:25.213095 19296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } }
I20260808 10:11:25.213414 19296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7102975c4a3946a5a00f9b9837293022, State: Initialized, Role: FOLLOWER
I20260808 10:11:25.213624 19296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } }
I20260808 10:11:25.213732 19296 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:11:25.213793 19296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:11:25.213846 19296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 2 FOLLOWER]: Advancing to term 3
I20260808 10:11:25.215133 19296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } }
I20260808 10:11:25.215276 19296 leader_election.cc:302] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7102975c4a3946a5a00f9b9837293022; no voters:
I20260808 10:11:25.215484 19296 leader_election.cc:288] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 10:11:25.215565 19301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 10:11:25.215794 19296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:11:25.215828 19301 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [term 3 LEADER]: Becoming Leader. State: Replica: 7102975c4a3946a5a00f9b9837293022, State: Running, Role: LEADER
I20260808 10:11:25.216079 19301 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } }
I20260808 10:11:25.216506 19302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7102975c4a3946a5a00f9b9837293022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } } }
I20260808 10:11:25.216604 19302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:25.216760 19303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7102975c4a3946a5a00f9b9837293022. Latest consensus state: current_term: 3 leader_uuid: "7102975c4a3946a5a00f9b9837293022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7102975c4a3946a5a00f9b9837293022" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38189 } } }
I20260808 10:11:25.216806 19303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022 [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:25.216976 19310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:11:25.217970 19310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:11:25.218170 19310 catalog_manager.cc:1259] Loaded cluster ID: 3999e8f231ed49bd8d5f6647b2b0f3fa
I20260808 10:11:25.218372 19310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:11:25.219969 19310 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:11:25.220328 19310 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Loaded TSK: 0
I20260808 10:11:25.220417 19310 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7102975c4a3946a5a00f9b9837293022: Loaded TSK: 1
F20260808 10:11:25.220779 19310 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f807f5809ad google::LogMessage::Fail() at ??:0
@ 0x7f807f58388a google::LogMessage::SendToLog() at ??:0
@ 0x7f807f5804de google::LogMessage::Flush() at ??:0
@ 0x7f807f581a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8081ced104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f8081cead41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f80874de1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f80874de48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f80874de594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f80874a4c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f80874aead0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f808144e0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8081443954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f80822a76db start_thread at ??:0
@ 0x7f807e07e61f clone at ??:0
W20260808 10:11:25.343528 19298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:25.343792 19298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:25.343833 19298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:25.345559 19298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:25.345661 19298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:25.345696 19298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:11:25.348696 19298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:44791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=34293
--tserver_master_addrs=127.2.30.190:38189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:25.349252 19298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:25.349653 19298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:25.357882 19339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:25.357879 19343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:25.357910 19341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:25.358781 19298 server_base.cc:1034] running on GCE node
I20260808 10:11:25.359203 19298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:25.359548 19298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:25.360747 19298 hybrid_clock.cc:648] HybridClock initialized: now 1786183885360728 us; error 56 us; skew 500 ppm
Aug 08 10:11:25 dist-test-slave-w206 krb5kdc[18110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183885, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:25.364902 19298 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:11:25.365536 19298 webserver.cc:460] Webserver started at http://127.2.30.129:34293/ using document root <none> and password file <none>
I20260808 10:11:25.365859 19298 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:25.365972 19298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:25.367726 19298 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:11:25.368708 19350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:25.369626 19298 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:25.369877 19298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "1635c9eb26f14513b0ec873c65662bad"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "d751ffd5da5b33481a9b2f7db24a73fb"
server_key_iv: "63621ec6b71fc1e7781dffd591b6d0ac"
server_key_version: "encryptionkey@0"
I20260808 10:11:25.370333 19298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:25.386876 19298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:25.389957 19298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:25.390125 19298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:25.390482 19298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:25.391062 19357 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 10:11:25.393548 19298 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 10:11:25.393640 19298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260808 10:11:25.393698 19298 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 10:11:25.394562 19298 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 10:11:25.394629 19298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 10:11:25.394707 19357 tablet_bootstrap.cc:492] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad: Bootstrap starting.
I20260808 10:11:25.403155 19298 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:44791
I20260808 10:11:25.403345 19472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:44791 every 8 connection(s)
I20260808 10:11:25.403683 19298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:11:25.408160 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 19298
I20260808 10:11:25.408305 19357 log.cc:824] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:25.408535 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:36317
--local_ip_for_outbound_sockets=127.2.30.130
--tserver_master_addrs=127.2.30.190:38189
--webserver_port=33171
--webserver_interface=127.2.30.130
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:39861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:11:25.413509 19357 tablet_bootstrap.cc:492] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 10:11:25.414000 19357 tablet_bootstrap.cc:492] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad: Bootstrap complete.
I20260808 10:11:25.415472 19357 ts_tablet_manager.cc:1397] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.013s
I20260808 10:11:25.417228 19357 raft_consensus.cc:348] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:25.417630 19357 raft_consensus.cc:729] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1635c9eb26f14513b0ec873c65662bad, State: Initialized, Role: FOLLOWER
I20260808 10:11:25.417832 19357 consensus_queue.cc:260] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:25.418177 19357 ts_tablet_manager.cc:1428] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
W20260808 10:11:25.498663 19475 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:11:25.404627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:50503)
0808 10:11:25.404929 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:25.404975 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:25.405505 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:25.498415 (+ 92910us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":142,"threads_started":1}
W20260808 10:11:25.498962 19473 heartbeater.cc:644] Failed to heartbeat to 127.2.30.190:38189 (0 consecutive failures): Network error: Failed to ping master at 127.2.30.190:38189: Client connection negotiation failed: client connection to 127.2.30.190:38189: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 10:11:25.499495 19475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:25.499299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:43405)
0808 10:11:25.499418 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:25.499469 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260808 10:11:25.499873 19475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:25.499776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:33571)
0808 10:11:25.499821 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:25.499847 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 10:11:25.500010 19473 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 10:11:25.508039 19477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:25.508390 19477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:25.508438 19477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:25.509924 19477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:25.510002 19477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:25.510056 19477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:11:25.513754 19477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:36317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=33171
--tserver_master_addrs=127.2.30.190:38189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:25.514142 19477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:25.514426 19477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:25.521287 19483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:25.521323 19477 server_base.cc:1034] running on GCE node
W20260808 10:11:25.521277 19486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:25.521277 19484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:25.521936 19477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:25.522233 19477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:25.523551 19477 hybrid_clock.cc:648] HybridClock initialized: now 1786183885523518 us; error 37 us; skew 500 ppm
Aug 08 10:11:25 dist-test-slave-w206 krb5kdc[18110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183885, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:25.527256 19477 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:11:25.527817 19477 webserver.cc:460] Webserver started at http://127.2.30.130:33171/ using document root <none> and password file <none>
I20260808 10:11:25.528087 19477 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:25.528163 19477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:25.529763 19477 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:11:25.530763 19493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:25.531196 19477 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:25.531323 19477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "379856a2bbad46dbaee75d1d406a1514"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "259c2784db517bc7bb66994448a68394"
server_key_iv: "801b17b01b979473fd1ea23c90182e28"
server_key_version: "encryptionkey@0"
I20260808 10:11:25.531793 19477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:25.549013 19477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:25.551651 19477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:25.551854 19477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:25.552143 19477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:25.552491 19477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:25.552544 19477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:25.552588 19477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:25.552632 19477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:25.559450 19477 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:36317
I20260808 10:11:25.559514 19611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:36317 every 8 connection(s)
I20260808 10:11:25.559929 19477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:11:25.561301 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:25.560756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:46869)
0808 10:11:25.560987 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:25.561072 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":125,"threads_started":1}
W20260808 10:11:25.561771 19612 heartbeater.cc:644] Failed to heartbeat to 127.2.30.190:38189 (0 consecutive failures): Network error: Failed to ping master at 127.2.30.190:38189: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
I20260808 10:11:25.562134 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:25.562023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:35881)
0808 10:11:25.562077 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:25.562112 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 10:11:25.562413 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:25.562331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:49261)
0808 10:11:25.562372 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:25.562394 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 10:11:25.562520 19612 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:11:25.566907 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 19477
I20260808 10:11:25.567340 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:35927
--local_ip_for_outbound_sockets=127.2.30.131
--tserver_master_addrs=127.2.30.190:38189
--webserver_port=37433
--webserver_interface=127.2.30.131
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:39861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:25.660669 19615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:25.660989 19615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:25.661038 19615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:25.662412 19615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:25.662503 19615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:25.662554 19615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:11:25.664367 19615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:39861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:35927
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=37433
--tserver_master_addrs=127.2.30.190:38189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:25.664700 19615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:25.664986 19615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:25.670531 19620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:25.670605 19621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:25.670539 19623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:25.670948 19615 server_base.cc:1034] running on GCE node
I20260808 10:11:25.671207 19615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:25.671455 19615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:25.672619 19615 hybrid_clock.cc:648] HybridClock initialized: now 1786183885672598 us; error 38 us; skew 500 ppm
Aug 08 10:11:25 dist-test-slave-w206 krb5kdc[18110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183885, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:25.675997 19615 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:11:25.676558 19615 webserver.cc:460] Webserver started at http://127.2.30.131:37433/ using document root <none> and password file <none>
I20260808 10:11:25.676839 19615 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:25.676911 19615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:25.678417 19615 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 10:11:25.679193 19631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:25.679394 19615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:25.679497 19615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "b896e6cb7fa74813a48e87cd76098119"
format_stamp: "Formatted at 2026-08-08 10:10:43 on dist-test-slave-w206"
server_key: "fb98b0dc6592c56fee9441b659c543d1"
server_key_iv: "53b4d90ed7cc6f94d830e71c7c6edc8c"
server_key_version: "encryptionkey@0"
I20260808 10:11:25.679912 19615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:25.694538 19615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:25.697482 19615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:25.697700 19615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:25.697965 19615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:25.698313 19615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:25.698369 19615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:25.698411 19615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:25.698442 19615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:25.704990 19615 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:35927
I20260808 10:11:25.705034 19749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:35927 every 8 connection(s)
I20260808 10:11:25.705415 19615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:11:25.706568 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:25.706120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:51143)
0808 10:11:25.706277 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:25.706396 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
W20260808 10:11:25.707044 19750 heartbeater.cc:644] Failed to heartbeat to 127.2.30.190:38189 (0 consecutive failures): Network error: Failed to ping master at 127.2.30.190:38189: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
I20260808 10:11:25.707335 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:25.707245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:60085)
0808 10:11:25.707288 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:25.707317 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 10:11:25.707647 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:25.707576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:42895)
0808 10:11:25.707612 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:25.707634 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 10:11:25.707758 19750 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 10:11:25.715063 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 19615
I20260808 10:11:26.500735 19755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:26.500478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:42597)
0808 10:11:26.500622 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:26.500666 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260808 10:11:26.563089 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:26.562843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:60265)
0808 10:11:26.562954 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:26.563031 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260808 10:11:26.579694 19757 raft_consensus.cc:482] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:11:26.579823 19757 raft_consensus.cc:504] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:26.580168 19757 leader_election.cc:288] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b896e6cb7fa74813a48e87cd76098119 (127.2.30.131:35927), 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317)
Aug 08 10:11:26 dist-test-slave-w206 krb5kdc[18110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183885, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.130@KRBTEST.COM
Aug 08 10:11:26 dist-test-slave-w206 krb5kdc[18110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183885, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.131@KRBTEST.COM
I20260808 10:11:26.591921 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:26.580472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:36317 (local address 127.2.30.129:36181)
0808 10:11:26.580603 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:26.580613 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:26.580670 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:26.581487 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:26.581488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:26.581949 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:26.582190 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:26.582197 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:26.583609 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:26.583610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:26.584487 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:26.584490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:26.585157 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:26.585158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:26.585255 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:26.585940 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:11:26.585949 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:26.587903 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:26.590192 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:26.590203 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:26.590205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:26.590494 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:26.590793 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:26.590793 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:26.590794 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:26.590907 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:26.591445 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:26.591451 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:26.591593 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 10:11:26.591681 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 10:11:26.591842 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":32,"thread_start_us":67,"threads_started":1}
I20260808 10:11:26.592058 19755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:26.580337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:35927 (local address 127.2.30.129:58077)
0808 10:11:26.580405 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:26.580421 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:26.580506 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:26.581482 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:26.581486 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:26.581956 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:26.582189 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:26.582197 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:26.583427 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:26.583428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:26.584488 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:26.584491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:26.585122 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:26.585123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:26.585253 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:26.585989 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:11:26.585996 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:26.588340 (+ 2344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:26.590373 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:26.590374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:26.590375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:26.590590 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:26.590886 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:26.590887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:26.590888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:26.590962 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:26.591517 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:26.591518 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:26.591595 (+ 77us) client_negotiation.cc:769] Sending connection context
0808 10:11:26.591699 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 10:11:26.591943 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":157}
I20260808 10:11:26.592073 19759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:26.580499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:58077 (local address 127.2.30.131:35927)
0808 10:11:26.580647 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:26.580659 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:26.580687 (+ 28us) server_negotiation.cc:407] Connection header received
0808 10:11:26.581217 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:26.581220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:26.581274 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:26.581455 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:26.582324 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:26.583318 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:26.584591 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:26.585025 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:26.585126 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:26.588479 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:26.588484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:26.588488 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:26.588499 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:26.590190 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:26.590699 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:26.590703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:26.590703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:26.590807 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:26.591062 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:26.591063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:26.591063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:26.591413 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:26.591528 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:26.591788 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 10:11:26.591954 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":61,"threads_started":1}
I20260808 10:11:26.592526 19760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:26.580576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:36181 (local address 127.2.30.130:36317)
0808 10:11:26.580744 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:26.580756 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:26.580794 (+ 38us) server_negotiation.cc:407] Connection header received
0808 10:11:26.581270 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:26.581274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:26.581324 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:26.581493 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:26.582314 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:26.583508 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:26.584572 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:26.585047 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:26.585197 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:26.588051 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:26.588057 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:26.588057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:26.588069 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:26.590077 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:26.590595 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:26.590598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:26.590599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:26.590699 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:26.591016 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:26.591017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:26.591018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:26.591327 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:26.591412 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:26.592267 (+ 855us) server_negotiation.cc:299] Negotiation successful
0808 10:11:26.592392 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
W20260808 10:11:26.593220 19353 leader_election.cc:341] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b896e6cb7fa74813a48e87cd76098119 (127.2.30.131:35927): Not found: Tablet not found: fdb8a12aeadd4249a9a825f9e65d22fe
W20260808 10:11:26.593303 19353 leader_election.cc:341] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317): Not found: Tablet not found: fdb8a12aeadd4249a9a825f9e65d22fe
I20260808 10:11:26.593331 19353 leader_election.cc:302] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1635c9eb26f14513b0ec873c65662bad; no voters: 379856a2bbad46dbaee75d1d406a1514, b896e6cb7fa74813a48e87cd76098119
I20260808 10:11:26.593432 19757 raft_consensus.cc:2726] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:11:26.708441 19761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:26.708122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:59987)
0808 10:11:26.708338 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:26.708398 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":138,"threads_started":1}
I20260808 10:11:27.501538 19762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:27.501292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:49175)
0808 10:11:27.501452 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:27.501486 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260808 10:11:27.564010 19763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:27.563694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:56199)
0808 10:11:27.563866 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:27.563943 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":69,"threads_started":1}
I20260808 10:11:27.709379 19764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:27.709041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:58941)
0808 10:11:27.709212 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:27.709308 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
I20260808 10:11:28.502323 19765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:28.502091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:36029)
0808 10:11:28.502227 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:28.502268 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260808 10:11:28.564883 19766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:28.564618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:38343)
0808 10:11:28.564780 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:28.564833 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":74,"threads_started":1}
I20260808 10:11:28.685401 19767 raft_consensus.cc:482] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:11:28.685509 19767 raft_consensus.cc:504] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:28.685736 19767 leader_election.cc:288] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b896e6cb7fa74813a48e87cd76098119 (127.2.30.131:35927), 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317)
W20260808 10:11:28.686429 19353 leader_election.cc:341] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b896e6cb7fa74813a48e87cd76098119 (127.2.30.131:35927): Not found: Tablet not found: fdb8a12aeadd4249a9a825f9e65d22fe
W20260808 10:11:28.686625 19353 leader_election.cc:341] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317): Not found: Tablet not found: fdb8a12aeadd4249a9a825f9e65d22fe
I20260808 10:11:28.686677 19353 leader_election.cc:302] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1635c9eb26f14513b0ec873c65662bad; no voters: 379856a2bbad46dbaee75d1d406a1514, b896e6cb7fa74813a48e87cd76098119
I20260808 10:11:28.686766 19767 raft_consensus.cc:2726] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:11:28.710280 19768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:28.710000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:42071)
0808 10:11:28.710173 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:28.710227 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":74,"threads_started":1}
I20260808 10:11:29.504299 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:29.503631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:46615)
0808 10:11:29.504050 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:29.504204 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":148,"threads_started":1}
I20260808 10:11:29.565884 19770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:29.565572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:34693)
0808 10:11:29.565780 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:29.565832 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":99,"threads_started":1}
I20260808 10:11:29.711333 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:29.711086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:36865)
0808 10:11:29.711242 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:29.711281 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
I20260808 10:11:30.505448 19772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:30.505146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:43107)
0808 10:11:30.505337 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:30.505392 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":72,"threads_started":1}
I20260808 10:11:30.534111 19773 raft_consensus.cc:482] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:11:30.534281 19773 raft_consensus.cc:504] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:30.534520 19773 leader_election.cc:288] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b896e6cb7fa74813a48e87cd76098119 (127.2.30.131:35927), 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317)
W20260808 10:11:30.535109 19353 leader_election.cc:341] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317): Not found: Tablet not found: fdb8a12aeadd4249a9a825f9e65d22fe
W20260808 10:11:30.535194 19353 leader_election.cc:341] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b896e6cb7fa74813a48e87cd76098119 (127.2.30.131:35927): Not found: Tablet not found: fdb8a12aeadd4249a9a825f9e65d22fe
I20260808 10:11:30.535241 19353 leader_election.cc:302] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1635c9eb26f14513b0ec873c65662bad; no voters: 379856a2bbad46dbaee75d1d406a1514, b896e6cb7fa74813a48e87cd76098119
I20260808 10:11:30.535336 19773 raft_consensus.cc:2726] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:11:30.566828 19774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:30.566532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:40433)
0808 10:11:30.566727 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:30.566776 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":94,"threads_started":1}
I20260808 10:11:30.712273 19775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:30.711995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:49479)
0808 10:11:30.712162 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:30.712212 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":88,"threads_started":1}
I20260808 10:11:31.506336 19776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:31.506013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:53105)
0808 10:11:31.506240 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:31.506284 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":94,"threads_started":1}
I20260808 10:11:31.567633 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:31.567400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:53677)
0808 10:11:31.567534 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:31.567581 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260808 10:11:31.713122 19778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:31.712867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:59179)
0808 10:11:31.713020 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:31.713067 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":70,"threads_started":1}
I20260808 10:11:32.507421 19779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:32.507099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:54149)
0808 10:11:32.507275 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:32.507321 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":89,"threads_started":1}
I20260808 10:11:32.568466 19780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:32.568197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:35365)
0808 10:11:32.568373 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:32.568416 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":94,"threads_started":1}
I20260808 10:11:32.713855 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:32.713613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:35813)
0808 10:11:32.713765 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:32.713802 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":65,"threads_started":1}
I20260808 10:11:33.508276 19782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:33.507999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:53005)
0808 10:11:33.508177 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:33.508223 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
I20260808 10:11:33.569171 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:33.568965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:47779)
0808 10:11:33.569084 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:33.569116 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260808 10:11:33.655488 19784 raft_consensus.cc:482] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:11:33.655599 19784 raft_consensus.cc:504] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:33.655740 19784 leader_election.cc:288] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b896e6cb7fa74813a48e87cd76098119 (127.2.30.131:35927), 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317)
W20260808 10:11:33.656291 19353 leader_election.cc:341] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b896e6cb7fa74813a48e87cd76098119 (127.2.30.131:35927): Not found: Tablet not found: fdb8a12aeadd4249a9a825f9e65d22fe
W20260808 10:11:33.656358 19353 leader_election.cc:341] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317): Not found: Tablet not found: fdb8a12aeadd4249a9a825f9e65d22fe
I20260808 10:11:33.656373 19353 leader_election.cc:302] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1635c9eb26f14513b0ec873c65662bad; no voters: 379856a2bbad46dbaee75d1d406a1514, b896e6cb7fa74813a48e87cd76098119
I20260808 10:11:33.656569 19784 raft_consensus.cc:2726] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:11:33.714807 19785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:33.714531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:40917)
0808 10:11:33.714695 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:33.714753 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":73,"threads_started":1}
I20260808 10:11:34.509066 19786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:34.508788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:34017)
0808 10:11:34.508965 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:34.509012 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":111,"threads_started":1}
I20260808 10:11:34.569994 19787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:34.569723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:51071)
0808 10:11:34.569894 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:34.569941 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":78,"threads_started":1}
I20260808 10:11:34.715631 19788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:34.715391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:47473)
0808 10:11:34.715534 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:34.715579 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260808 10:11:35.509897 19789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:35.509638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:38285)
0808 10:11:35.509800 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:35.509848 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":67,"threads_started":1}
I20260808 10:11:35.570847 19790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:35.570567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:56143)
0808 10:11:35.570751 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:35.570795 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":88,"threads_started":1}
I20260808 10:11:35.716454 19791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:35.716192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:57889)
0808 10:11:35.716349 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:35.716402 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
I20260808 10:11:36.192706 19792 raft_consensus.cc:482] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:11:36.192865 19792 raft_consensus.cc:504] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b896e6cb7fa74813a48e87cd76098119" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35927 } } peers { permanent_uuid: "379856a2bbad46dbaee75d1d406a1514" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36317 } } peers { permanent_uuid: "1635c9eb26f14513b0ec873c65662bad" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 44791 } }
I20260808 10:11:36.193117 19792 leader_election.cc:288] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b896e6cb7fa74813a48e87cd76098119 (127.2.30.131:35927), 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317)
W20260808 10:11:36.193730 19353 leader_election.cc:341] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b896e6cb7fa74813a48e87cd76098119 (127.2.30.131:35927): Not found: Tablet not found: fdb8a12aeadd4249a9a825f9e65d22fe
W20260808 10:11:36.193821 19353 leader_election.cc:341] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 379856a2bbad46dbaee75d1d406a1514 (127.2.30.130:36317): Not found: Tablet not found: fdb8a12aeadd4249a9a825f9e65d22fe
I20260808 10:11:36.193882 19353 leader_election.cc:302] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1635c9eb26f14513b0ec873c65662bad; no voters: 379856a2bbad46dbaee75d1d406a1514, b896e6cb7fa74813a48e87cd76098119
I20260808 10:11:36.194017 19792 raft_consensus.cc:2726] T fdb8a12aeadd4249a9a825f9e65d22fe P 1635c9eb26f14513b0ec873c65662bad [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 10:11:36.510799 19793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:36.510517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:46423)
0808 10:11:36.510700 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:36.510748 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":96,"threads_started":1}
I20260808 10:11:36.571606 19794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:36.571362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:44265)
0808 10:11:36.571517 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:36.571557 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260808 10:11:36.717242 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:36.717001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:58401)
0808 10:11:36.717143 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:36.717190 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260808 10:11:37.511663 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:37.511399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:45715)
0808 10:11:37.511558 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:37.511612 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260808 10:11:37.572415 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:37.572149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:34345)
0808 10:11:37.572312 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:37.572359 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
I20260808 10:11:37.718185 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:37.717869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:43691)
0808 10:11:37.718065 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:37.718130 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260808 10:11:38.512537 19799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:38.512262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:60193)
0808 10:11:38.512440 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:38.512487 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
I20260808 10:11:38.573115 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:38.572903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:52161)
0808 10:11:38.573029 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:38.573065 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260808 10:11:38.719094 19801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:38.718747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:51053)
0808 10:11:38.718930 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:38.719040 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":71,"threads_started":1}
I20260808 10:11:39.513401 19802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:39.513127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:55123)
0808 10:11:39.513296 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:39.513339 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":74,"threads_started":1}
I20260808 10:11:39.573873 19803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:39.573648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:45619)
0808 10:11:39.573787 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:39.573822 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260808 10:11:39.719924 19804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:39.719599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:55467)
0808 10:11:39.719806 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:39.719850 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":122,"threads_started":1}
I20260808 10:11:40.514451 19805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:40.514179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.129:51323)
0808 10:11:40.514349 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:40.514394 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":94,"threads_started":1}
I20260808 10:11:40.574783 19806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:40.574552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.130:32783)
0808 10:11:40.574682 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:40.574731 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260808 10:11:40.720934 19807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:40.720664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38189 (local address 127.2.30.131:41735)
0808 10:11:40.720834 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:40.720882 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.190:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":75,"threads_started":1}
I20260808 10:11:40.723902 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 19298
I20260808 10:11:40.733228 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 19477
I20260808 10:11:40.740610 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 19615
2026-08-08T10:11:40Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59117 ms)
[----------] 30 tests from SecurityITest (249071 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:11:40 dist-test-slave-w206 krb5kdc[19810](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:11:40 dist-test-slave-w206 krb5kdc[19810](info): set up 2 sockets
Aug 08 10:11:40 dist-test-slave-w206 krb5kdc[19810](info): commencing operation
krb5kdc: starting...
W20260808 10:11:41.856441 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:11:41 dist-test-slave-w206 krb5kdc[19810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:11:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:11:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:11:41.994804 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:40819
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:34685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:40819
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:42.080667 19823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:42.080878 19823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:42.080901 19823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:42.081990 19823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:11:42.082024 19823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:42.082039 19823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:42.082052 19823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:11:42.082064 19823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:11:42.083531 19823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:34685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:40819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:40819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:42.083727 19823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:42.083904 19823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:42.089288 19829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:42.089275 19830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:42.089272 19823 server_base.cc:1034] running on GCE node
W20260808 10:11:42.089262 19832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:42.089813 19823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:42.090094 19823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:42.091253 19823 hybrid_clock.cc:648] HybridClock initialized: now 1786183902091191 us; error 78 us; skew 500 ppm
Aug 08 10:11:42 dist-test-slave-w206 krb5kdc[19810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183902, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:42.093937 19823 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:11:42.094376 19823 webserver.cc:460] Webserver started at http://127.2.30.190:34579/ using document root <none> and password file <none>
I20260808 10:11:42.094619 19823 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:42.094691 19823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:42.094851 19823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:42.095999 19823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "f3cbadad8b3e42abbce611a8df82ee97"
format_stamp: "Formatted at 2026-08-08 10:11:42 on dist-test-slave-w206"
server_key: "cc59e36cb945121ba33d7b74e4214ffd"
server_key_iv: "f99b98a8282b6b76437fb844877a0d2f"
server_key_version: "encryptionkey@0"
I20260808 10:11:42.096416 19823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "f3cbadad8b3e42abbce611a8df82ee97"
format_stamp: "Formatted at 2026-08-08 10:11:42 on dist-test-slave-w206"
server_key: "cc59e36cb945121ba33d7b74e4214ffd"
server_key_iv: "f99b98a8282b6b76437fb844877a0d2f"
server_key_version: "encryptionkey@0"
I20260808 10:11:42.097762 19823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:11:42.098510 19839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:42.098690 19823 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 10:11:42.098793 19823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "f3cbadad8b3e42abbce611a8df82ee97"
format_stamp: "Formatted at 2026-08-08 10:11:42 on dist-test-slave-w206"
server_key: "cc59e36cb945121ba33d7b74e4214ffd"
server_key_iv: "f99b98a8282b6b76437fb844877a0d2f"
server_key_version: "encryptionkey@0"
I20260808 10:11:42.098877 19823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:42.111512 19823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:42.113976 19823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:42.114149 19823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:42.118525 19823 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:40819
I20260808 10:11:42.118571 19892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:40819 every 8 connection(s)
I20260808 10:11:42.118932 19823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:11:42.119973 19893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:42.120810 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 19823
I20260808 10:11:42.120927 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:11:42.121129 2170 external_mini_cluster.cc:1428] Setting key e673c946936f38318917515ece0b65d7
I20260808 10:11:42.123147 19893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97: Bootstrap starting.
I20260808 10:11:42.123744 19893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:42.124056 19893 log.cc:824] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:42.124876 19893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97: No bootstrap required, opened a new log
I20260808 10:11:42.127451 19893 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cbadad8b3e42abbce611a8df82ee97" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 40819 } }
I20260808 10:11:42.127637 19893 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:42.127681 19893 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3cbadad8b3e42abbce611a8df82ee97, State: Initialized, Role: FOLLOWER
I20260808 10:11:42.127835 19893 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cbadad8b3e42abbce611a8df82ee97" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 40819 } }
I20260808 10:11:42.127934 19893 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:11:42.127967 19893 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:11:42.128016 19893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 10:11:42 dist-test-slave-w206 krb5kdc[19810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:11:42.128763 19893 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cbadad8b3e42abbce611a8df82ee97" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 40819 } }
I20260808 10:11:42.128890 19893 leader_election.cc:302] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3cbadad8b3e42abbce611a8df82ee97; no voters:
I20260808 10:11:42.129096 19893 leader_election.cc:288] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:11:42.129148 19898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:42.129333 19898 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [term 1 LEADER]: Becoming Leader. State: Replica: f3cbadad8b3e42abbce611a8df82ee97, State: Running, Role: LEADER
I20260808 10:11:42.129386 19893 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:11:42.129544 19898 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cbadad8b3e42abbce611a8df82ee97" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 40819 } }
I20260808 10:11:42.129947 19899 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3cbadad8b3e42abbce611a8df82ee97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cbadad8b3e42abbce611a8df82ee97" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 40819 } } }
I20260808 10:11:42.130033 19899 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:42.130266 19906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:11:42.131181 19900 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3cbadad8b3e42abbce611a8df82ee97. Latest consensus state: current_term: 1 leader_uuid: "f3cbadad8b3e42abbce611a8df82ee97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3cbadad8b3e42abbce611a8df82ee97" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 40819 } } }
I20260808 10:11:42.131290 19900 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97 [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:42.131884 19906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:11:42.133561 19896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.121712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37514 (local address 127.2.30.190:40819)
0808 10:11:42.121913 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:42.121922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:42.121957 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:11:42.122381 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:42.122395 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:42.122446 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:42.122597 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:42.123346 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.124439 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.125487 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.125893 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.125977 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.129061 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:42.129065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:42.129066 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:42.129078 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:42.131685 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:42.132178 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:42.132182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:42.132182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:42.132296 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:42.132566 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:42.132568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:42.132568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:42.132888 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:42.133068 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:42.133191 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 10:11:42.133356 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260808 10:11:42.134080 19906 catalog_manager.cc:1347] Generated new cluster ID: 8d77c6aaadf7474b93a9ee284627af53
I20260808 10:11:42.134135 19906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:11:42.143342 19906 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:11:42.144112 19906 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:11:42.151443 19906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97: Generated new TSK 0
I20260808 10:11:42.151690 19906 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:11:42.190392 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:40819
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:34685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:42.296516 19919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:42.296756 19919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:42.296799 19919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:42.297994 19919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:42.298060 19919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:42.298110 19919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:11:42.299736 19919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:34685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:40819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:42.300037 19919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:42.300287 19919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:42.306496 19924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:42.306502 19925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:42.306811 19919 server_base.cc:1034] running on GCE node
W20260808 10:11:42.306507 19927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:42.307153 19919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:42.307421 19919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:42.308593 19919 hybrid_clock.cc:648] HybridClock initialized: now 1786183902308572 us; error 35 us; skew 500 ppm
Aug 08 10:11:42 dist-test-slave-w206 krb5kdc[19810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183902, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:42.311750 19919 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:11:42.312269 19919 webserver.cc:460] Webserver started at http://127.2.30.129:40863/ using document root <none> and password file <none>
I20260808 10:11:42.312517 19919 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:42.312587 19919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:42.312736 19919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:42.313807 19919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "c75baaa85e564c8c860d25bdb13e0432"
format_stamp: "Formatted at 2026-08-08 10:11:42 on dist-test-slave-w206"
server_key: "e45c7c50823e664873422f8a62840222"
server_key_iv: "bb00bd8f6f8aa85cb5766af834ed9198"
server_key_version: "encryptionkey@0"
I20260808 10:11:42.314222 19919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "c75baaa85e564c8c860d25bdb13e0432"
format_stamp: "Formatted at 2026-08-08 10:11:42 on dist-test-slave-w206"
server_key: "e45c7c50823e664873422f8a62840222"
server_key_iv: "bb00bd8f6f8aa85cb5766af834ed9198"
server_key_version: "encryptionkey@0"
I20260808 10:11:42.315709 19919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:11:42.316491 19934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:42.316675 19919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:11:42.316781 19919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "c75baaa85e564c8c860d25bdb13e0432"
format_stamp: "Formatted at 2026-08-08 10:11:42 on dist-test-slave-w206"
server_key: "e45c7c50823e664873422f8a62840222"
server_key_iv: "bb00bd8f6f8aa85cb5766af834ed9198"
server_key_version: "encryptionkey@0"
I20260808 10:11:42.316855 19919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:42.335620 19919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:42.337823 19919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:42.338007 19919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:42.338266 19919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:42.338600 19919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:42.338660 19919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:42.338703 19919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:42.338732 19919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:42.345500 19919 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:36309
I20260808 10:11:42.345554 20052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:36309 every 8 connection(s)
I20260808 10:11:42.345906 19919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
Aug 08 10:11:42 dist-test-slave-w206 krb5kdc[19810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183902, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:11:42.354893 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 19919
I20260808 10:11:42.355058 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:11:42.355270 2170 external_mini_cluster.cc:1428] Setting key ce76567aa8144c62596805a048ae2808
I20260808 10:11:42.357136 19896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.346763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:48257 (local address 127.2.30.190:40819)
0808 10:11:42.346855 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:42.346858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:42.347105 (+ 247us) server_negotiation.cc:407] Connection header received
0808 10:11:42.347778 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:42.347780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:42.347794 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:42.347857 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:42.348950 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.349510 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.350661 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.350885 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.351004 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.353566 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:42.353568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:42.353568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:42.353578 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:42.355392 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:42.355966 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:42.355967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:42.355967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:42.356046 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:42.356353 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:42.356354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:42.356354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:42.356496 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:42.356588 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:42.356911 (+ 323us) server_negotiation.cc:299] Negotiation successful
0808 10:11:42.357030 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 10:11:42.357370 20055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.346636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:40819 (local address 127.2.30.129:48257)
0808 10:11:42.346902 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:42.346960 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:42.347638 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:42.347930 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:42.347935 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:42.348599 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:42.348819 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:42.348825 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.349637 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.349638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:42.350541 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:42.350544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.351026 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.351027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:42.351186 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.351858 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:11:42.351866 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:42.353446 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:42.355500 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:42.355511 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:42.355514 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:42.355865 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:42.356126 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:42.356127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:42.356127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:42.356271 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:42.356576 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:42.356585 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:42.356758 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 10:11:42.356887 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 10:11:42.357084 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":114,"threads_started":1}
I20260808 10:11:42.358537 20053 heartbeater.cc:347] Connected to a master server at 127.2.30.190:40819
I20260808 10:11:42.358651 20053 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:42.358917 20053 heartbeater.cc:511] Master 127.2.30.190:40819 requested a full tablet report, sending...
I20260808 10:11:42.359673 19857 ts_manager.cc:194] Registered new tserver with Master: c75baaa85e564c8c860d25bdb13e0432 (127.2.30.129:36309)
I20260808 10:11:42.360658 19857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:48257
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:11:42.385828 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:40819
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:34685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:42.470077 20059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:42.470293 20059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:42.470317 20059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:42.471531 20059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:42.471575 20059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:42.471603 20059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:11:42.473153 20059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:34685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:40819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:42.473428 20059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:42.473692 20059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:42.479204 20066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:42.479209 20067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:42.479204 20069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:42.479271 20059 server_base.cc:1034] running on GCE node
I20260808 10:11:42.479758 20059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:42.480036 20059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:42.481213 20059 hybrid_clock.cc:648] HybridClock initialized: now 1786183902481200 us; error 32 us; skew 500 ppm
Aug 08 10:11:42 dist-test-slave-w206 krb5kdc[19810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183902, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:42.484032 20059 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:11:42.484647 20059 webserver.cc:460] Webserver started at http://127.2.30.130:42039/ using document root <none> and password file <none>
I20260808 10:11:42.484908 20059 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:42.484979 20059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:42.485140 20059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:42.486383 20059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "7e5dee9311694229ac852a082eb5b523"
format_stamp: "Formatted at 2026-08-08 10:11:42 on dist-test-slave-w206"
server_key: "f04b8a85a58d310e2b6a30f960efcffb"
server_key_iv: "e52dcef481e8211d2a2a268907d108f8"
server_key_version: "encryptionkey@0"
I20260808 10:11:42.486786 20059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "7e5dee9311694229ac852a082eb5b523"
format_stamp: "Formatted at 2026-08-08 10:11:42 on dist-test-slave-w206"
server_key: "f04b8a85a58d310e2b6a30f960efcffb"
server_key_iv: "e52dcef481e8211d2a2a268907d108f8"
server_key_version: "encryptionkey@0"
I20260808 10:11:42.488359 20059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:42.489243 20076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:42.489478 20059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:42.489586 20059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "7e5dee9311694229ac852a082eb5b523"
format_stamp: "Formatted at 2026-08-08 10:11:42 on dist-test-slave-w206"
server_key: "f04b8a85a58d310e2b6a30f960efcffb"
server_key_iv: "e52dcef481e8211d2a2a268907d108f8"
server_key_version: "encryptionkey@0"
I20260808 10:11:42.489672 20059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:42.521167 20059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:42.523337 20059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:42.523524 20059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:42.523797 20059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:42.524176 20059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:42.524245 20059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:42.524286 20059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:42.524317 20059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:42.530679 20059 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:40695
I20260808 10:11:42.530824 20196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:40695 every 8 connection(s)
I20260808 10:11:42.531159 20059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:11:42.532142 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 20059
I20260808 10:11:42.532285 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:11:42.532507 2170 external_mini_cluster.cc:1428] Setting key da61a0af8fa71b2401401ad34ac5e5d1
Aug 08 10:11:42 dist-test-slave-w206 krb5kdc[19810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183902, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
I20260808 10:11:42.542352 19896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.532025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:60109 (local address 127.2.30.190:40819)
0808 10:11:42.532080 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:42.532082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:42.532312 (+ 230us) server_negotiation.cc:407] Connection header received
0808 10:11:42.533020 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:42.533022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:42.533035 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:42.533078 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:42.534525 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.535138 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.536276 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.536525 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.536585 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.539359 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:42.539361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:42.539361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:42.539370 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:42.540835 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:42.541263 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:42.541264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:42.541265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:42.541328 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:42.541587 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:42.541588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:42.541589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:42.541714 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:42.541793 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:42.542096 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 10:11:42.542180 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 10:11:42.542544 20200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.531903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:40819 (local address 127.2.30.130:60109)
0808 10:11:42.532153 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:42.532212 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:42.532887 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:42.533170 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:42.533180 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:42.534079 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:42.534393 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:42.534401 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.535260 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.535261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:42.536177 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:42.536180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.536622 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.536624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:42.536777 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.537408 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:11:42.537419 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:42.539251 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:42.540924 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:42.540933 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:42.540935 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:42.541177 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:42.541400 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:42.541400 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:42.541401 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:42.541516 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:42.541798 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:42.541806 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:42.541952 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 10:11:42.542060 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 10:11:42.542237 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":108,"threads_started":1}
I20260808 10:11:42.543287 20198 heartbeater.cc:347] Connected to a master server at 127.2.30.190:40819
I20260808 10:11:42.543412 20198 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:42.543604 20198 heartbeater.cc:511] Master 127.2.30.190:40819 requested a full tablet report, sending...
I20260808 10:11:42.544008 19857 ts_manager.cc:194] Registered new tserver with Master: 7e5dee9311694229ac852a082eb5b523 (127.2.30.130:40695)
I20260808 10:11:42.544430 19857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:60109
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:11:42.561738 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:40819
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:34685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:42.647141 20203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:42.647373 20203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:42.647416 20203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:42.648641 20203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:42.648708 20203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:42.648754 20203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:11:42.650286 20203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:34685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:40819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:42.650563 20203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:42.650791 20203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:42.656255 20208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:42.656235 20211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:42.656354 20209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:42.656354 20203 server_base.cc:1034] running on GCE node
I20260808 10:11:42.656711 20203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:42.656944 20203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:42.658082 20203 hybrid_clock.cc:648] HybridClock initialized: now 1786183902658053 us; error 46 us; skew 500 ppm
Aug 08 10:11:42 dist-test-slave-w206 krb5kdc[19810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183902, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:42.660540 20203 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:11:42.660993 20203 webserver.cc:460] Webserver started at http://127.2.30.131:34601/ using document root <none> and password file <none>
I20260808 10:11:42.661228 20203 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:42.661294 20203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:42.661450 20203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:42.662496 20203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "227a12e2726444f6957ec87f8d1c9ce1"
format_stamp: "Formatted at 2026-08-08 10:11:42 on dist-test-slave-w206"
server_key: "b621c5976f83c694fcc83544596d7d6a"
server_key_iv: "74f7d0a162e8b6088d5586752545400b"
server_key_version: "encryptionkey@0"
I20260808 10:11:42.662861 20203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "227a12e2726444f6957ec87f8d1c9ce1"
format_stamp: "Formatted at 2026-08-08 10:11:42 on dist-test-slave-w206"
server_key: "b621c5976f83c694fcc83544596d7d6a"
server_key_iv: "74f7d0a162e8b6088d5586752545400b"
server_key_version: "encryptionkey@0"
I20260808 10:11:42.664244 20203 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:42.665056 20218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:42.665256 20203 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:42.665355 20203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "227a12e2726444f6957ec87f8d1c9ce1"
format_stamp: "Formatted at 2026-08-08 10:11:42 on dist-test-slave-w206"
server_key: "b621c5976f83c694fcc83544596d7d6a"
server_key_iv: "74f7d0a162e8b6088d5586752545400b"
server_key_version: "encryptionkey@0"
I20260808 10:11:42.665441 20203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:42.687881 20203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:42.692000 20203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:42.692210 20203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:42.692512 20203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:42.692957 20203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:42.692996 20203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:42.693019 20203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:42.693033 20203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:42.699378 20203 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:40401
I20260808 10:11:42.699545 20338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:40401 every 8 connection(s)
I20260808 10:11:42.699827 20203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
Aug 08 10:11:42 dist-test-slave-w206 krb5kdc[19810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183902, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:11:42.708032 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 20203
I20260808 10:11:42.708142 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:11:42.708335 2170 external_mini_cluster.cc:1428] Setting key 9c0befbd45a9ecbed6e21f6e73475740
I20260808 10:11:42.710457 19896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.700712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:44739 (local address 127.2.30.190:40819)
0808 10:11:42.700782 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:42.700785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:42.701049 (+ 264us) server_negotiation.cc:407] Connection header received
0808 10:11:42.701642 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:42.701644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:42.701658 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:42.701710 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:42.702779 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.703370 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.704382 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.704583 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.704662 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.706946 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:42.706947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:42.706947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:42.706956 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:42.708651 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:42.709390 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:42.709391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:42.709392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:42.709464 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:42.709770 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:42.709771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:42.709771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:42.709948 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:42.710024 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:42.710319 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 10:11:42.710392 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:11:42.710645 20341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.700715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:40819 (local address 127.2.30.131:44739)
0808 10:11:42.700918 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:42.700956 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:42.701519 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:42.701787 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:42.701790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:42.702461 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:42.702665 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:42.702670 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.703485 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.703486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:42.704277 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:42.704279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.704687 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.704688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:42.704825 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.705364 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:11:42.705372 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:42.706840 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:42.708804 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:42.708813 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:42.708815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:42.709207 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:42.709547 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:42.709547 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:42.709548 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:42.709692 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:42.710026 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:42.710038 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:42.710193 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 10:11:42.710288 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 10:11:42.710421 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":105,"threads_started":1}
I20260808 10:11:42.711215 20339 heartbeater.cc:347] Connected to a master server at 127.2.30.190:40819
I20260808 10:11:42.711335 20339 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:42.711534 20339 heartbeater.cc:511] Master 127.2.30.190:40819 requested a full tablet report, sending...
I20260808 10:11:42.712103 19857 ts_manager.cc:194] Registered new tserver with Master: 227a12e2726444f6957ec87f8d1c9ce1 (127.2.30.131:40401)
I20260808 10:11:42.712499 19857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:44739
I20260808 10:11:42.721091 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:11:42.729110 19896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.722593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37518 (local address 127.2.30.190:40819)
0808 10:11:42.722708 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:42.722709 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:42.722780 (+ 71us) server_negotiation.cc:407] Connection header received
0808 10:11:42.722859 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:42.722860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:42.722871 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:42.722921 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:42.723622 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.724120 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.724896 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.725084 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.725151 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.726453 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:42.726454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:42.726454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:42.726461 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:42.727833 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:42.728257 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:42.728258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:42.728258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:42.728319 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:42.728511 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:42.728511 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:42.728511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:42.728675 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:42.728767 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:42.728909 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 10:11:42.728996 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 10:11:42.731178 19857 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:11:42.733081 19857 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:11:42.740476 20354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.736130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:40401 (local address 127.0.0.1:57738)
0808 10:11:42.736994 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:42.737005 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:42.737063 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:42.737249 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:42.737250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:42.737252 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:42.737299 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:42.737301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.738411 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.738412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:42.739426 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:42.739428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.740139 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.740140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:42.740220 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.740234 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 10:11:42.740337 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 10:11:42.740396 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"thread_start_us":91,"threads_started":1}
I20260808 10:11:42.740489 20356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.736368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57738 (local address 127.2.30.131:40401)
0808 10:11:42.736594 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:42.736599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:42.737059 (+ 460us) server_negotiation.cc:407] Connection header received
0808 10:11:42.737129 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:42.737132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:42.737148 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:42.737229 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:42.737357 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.738304 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.739519 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.740037 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.740127 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.740200 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:42.740356 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 10:11:42.740398 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":97,"threads_started":1}
I20260808 10:11:42.740898 20355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.736129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39076 (local address 127.2.30.130:40695)
0808 10:11:42.736405 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:42.736410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:42.736421 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:11:42.736467 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:42.736470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:42.736485 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:42.736574 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:42.736794 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.737961 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.739548 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.740310 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.740385 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.740439 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:42.740774 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 10:11:42.740823 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":85,"threads_started":1}
I20260808 10:11:42.740947 20353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.736052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52258 (local address 127.2.30.129:36309)
0808 10:11:42.736718 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:42.736722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:42.736732 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:11:42.736772 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:42.736774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:42.736791 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:42.736870 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:42.737011 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.737872 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.739548 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.740310 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.740718 (+ 408us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.740793 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:42.740817 (+ 24us) server_negotiation.cc:299] Negotiation successful
0808 10:11:42.740862 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":86,"threads_started":1}
I20260808 10:11:42.740669 20351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.735968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:40695 (local address 127.0.0.1:39076)
0808 10:11:42.736222 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:42.736232 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:42.736314 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:42.736580 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:42.736583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:42.736586 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:42.736667 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:42.736669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.738063 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.738064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:42.739439 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:42.739441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.740468 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.740468 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:42.740522 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.740526 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:11:42.740571 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 10:11:42.740601 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":69,"thread_start_us":90,"threads_started":1}
I20260808 10:11:42.740669 20352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.735915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:36309 (local address 127.0.0.1:52258)
0808 10:11:42.736136 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:42.736180 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:42.736314 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:42.736870 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:42.736871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:42.736873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:42.736926 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:42.736927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.737983 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.737984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:42.739438 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:42.739449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.740415 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.740416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:42.740491 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.740494 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:11:42.740525 (+ 31us) client_negotiation.cc:240] Negotiation successful
0808 10:11:42.740577 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":40,"thread_start_us":75,"threads_started":1}
I20260808 10:11:42.741719 20270 tablet_service.cc:1467] Processing CreateTablet for tablet d4edd35174e148a1a4946d52adebfff9 (DEFAULT_TABLE table=test-table [id=62d08be9bfa34122ba0d1e589b65caa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:42.742115 20270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4edd35174e148a1a4946d52adebfff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:42.742142 20127 tablet_service.cc:1467] Processing CreateTablet for tablet d4edd35174e148a1a4946d52adebfff9 (DEFAULT_TABLE table=test-table [id=62d08be9bfa34122ba0d1e589b65caa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:42.742164 19984 tablet_service.cc:1467] Processing CreateTablet for tablet d4edd35174e148a1a4946d52adebfff9 (DEFAULT_TABLE table=test-table [id=62d08be9bfa34122ba0d1e589b65caa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:42.742424 19984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4edd35174e148a1a4946d52adebfff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:42.742447 20127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4edd35174e148a1a4946d52adebfff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:42.745805 20357 tablet_bootstrap.cc:492] T d4edd35174e148a1a4946d52adebfff9 P c75baaa85e564c8c860d25bdb13e0432: Bootstrap starting.
I20260808 10:11:42.745939 20359 tablet_bootstrap.cc:492] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523: Bootstrap starting.
I20260808 10:11:42.746141 20358 tablet_bootstrap.cc:492] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1: Bootstrap starting.
I20260808 10:11:42.746592 20357 tablet_bootstrap.cc:654] T d4edd35174e148a1a4946d52adebfff9 P c75baaa85e564c8c860d25bdb13e0432: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:42.746639 20359 tablet_bootstrap.cc:654] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:42.746953 20358 tablet_bootstrap.cc:654] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:42.747274 20357 log.cc:824] T d4edd35174e148a1a4946d52adebfff9 P c75baaa85e564c8c860d25bdb13e0432: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:42.747331 20359 log.cc:824] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:42.747610 20358 log.cc:824] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:42.748099 20357 tablet_bootstrap.cc:492] T d4edd35174e148a1a4946d52adebfff9 P c75baaa85e564c8c860d25bdb13e0432: No bootstrap required, opened a new log
I20260808 10:11:42.748158 20357 ts_tablet_manager.cc:1397] T d4edd35174e148a1a4946d52adebfff9 P c75baaa85e564c8c860d25bdb13e0432: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:11:42.748354 20358 tablet_bootstrap.cc:492] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1: No bootstrap required, opened a new log
I20260808 10:11:42.748438 20358 ts_tablet_manager.cc:1397] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 10:11:42.749807 20357 raft_consensus.cc:348] T d4edd35174e148a1a4946d52adebfff9 P c75baaa85e564c8c860d25bdb13e0432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75baaa85e564c8c860d25bdb13e0432" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36309 } } peers { permanent_uuid: "227a12e2726444f6957ec87f8d1c9ce1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40401 } } peers { permanent_uuid: "7e5dee9311694229ac852a082eb5b523" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40695 } }
I20260808 10:11:42.750030 20357 raft_consensus.cc:374] T d4edd35174e148a1a4946d52adebfff9 P c75baaa85e564c8c860d25bdb13e0432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:42.750080 20357 raft_consensus.cc:729] T d4edd35174e148a1a4946d52adebfff9 P c75baaa85e564c8c860d25bdb13e0432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75baaa85e564c8c860d25bdb13e0432, State: Initialized, Role: FOLLOWER
I20260808 10:11:42.750017 20358 raft_consensus.cc:348] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75baaa85e564c8c860d25bdb13e0432" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36309 } } peers { permanent_uuid: "227a12e2726444f6957ec87f8d1c9ce1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40401 } } peers { permanent_uuid: "7e5dee9311694229ac852a082eb5b523" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40695 } }
I20260808 10:11:42.750203 20357 consensus_queue.cc:260] T d4edd35174e148a1a4946d52adebfff9 P c75baaa85e564c8c860d25bdb13e0432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75baaa85e564c8c860d25bdb13e0432" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36309 } } peers { permanent_uuid: "227a12e2726444f6957ec87f8d1c9ce1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40401 } } peers { permanent_uuid: "7e5dee9311694229ac852a082eb5b523" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40695 } }
I20260808 10:11:42.750263 20358 raft_consensus.cc:374] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:42.750308 20358 raft_consensus.cc:729] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227a12e2726444f6957ec87f8d1c9ce1, State: Initialized, Role: FOLLOWER
I20260808 10:11:42.750422 20358 consensus_queue.cc:260] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75baaa85e564c8c860d25bdb13e0432" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36309 } } peers { permanent_uuid: "227a12e2726444f6957ec87f8d1c9ce1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40401 } } peers { permanent_uuid: "7e5dee9311694229ac852a082eb5b523" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40695 } }
I20260808 10:11:42.750521 20357 ts_tablet_manager.cc:1428] T d4edd35174e148a1a4946d52adebfff9 P c75baaa85e564c8c860d25bdb13e0432: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 10:11:42.750612 20053 heartbeater.cc:503] Master 127.2.30.190:40819 was elected leader, sending a full tablet report...
I20260808 10:11:42.750801 20358 ts_tablet_manager.cc:1428] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 10:11:42.750845 20339 heartbeater.cc:503] Master 127.2.30.190:40819 was elected leader, sending a full tablet report...
I20260808 10:11:42.751084 20359 tablet_bootstrap.cc:492] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523: No bootstrap required, opened a new log
I20260808 10:11:42.751154 20359 ts_tablet_manager.cc:1397] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 10:11:42.752897 20359 raft_consensus.cc:348] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75baaa85e564c8c860d25bdb13e0432" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36309 } } peers { permanent_uuid: "227a12e2726444f6957ec87f8d1c9ce1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40401 } } peers { permanent_uuid: "7e5dee9311694229ac852a082eb5b523" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40695 } }
I20260808 10:11:42.753073 20359 raft_consensus.cc:374] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:42.753121 20359 raft_consensus.cc:729] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e5dee9311694229ac852a082eb5b523, State: Initialized, Role: FOLLOWER
I20260808 10:11:42.753252 20359 consensus_queue.cc:260] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75baaa85e564c8c860d25bdb13e0432" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36309 } } peers { permanent_uuid: "227a12e2726444f6957ec87f8d1c9ce1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40401 } } peers { permanent_uuid: "7e5dee9311694229ac852a082eb5b523" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40695 } }
I20260808 10:11:42.753556 20359 ts_tablet_manager.cc:1428] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:11:42.753590 20198 heartbeater.cc:503] Master 127.2.30.190:40819 was elected leader, sending a full tablet report...
W20260808 10:11:42.781805 20199 tablet.cc:2369] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:11:42.806394 20365 raft_consensus.cc:482] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:11:42.806535 20365 raft_consensus.cc:504] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75baaa85e564c8c860d25bdb13e0432" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36309 } } peers { permanent_uuid: "227a12e2726444f6957ec87f8d1c9ce1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40401 } } peers { permanent_uuid: "7e5dee9311694229ac852a082eb5b523" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40695 } }
I20260808 10:11:42.807015 20365 leader_election.cc:288] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75baaa85e564c8c860d25bdb13e0432 (127.2.30.129:36309), 227a12e2726444f6957ec87f8d1c9ce1 (127.2.30.131:40401)
I20260808 10:11:42.810284 20200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.807305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:36309 (local address 127.2.30.130:48843)
0808 10:11:42.807374 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:42.807387 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:42.807437 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:42.807695 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:42.807696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:42.807698 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:42.807805 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:42.807808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.808557 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.808558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:42.809397 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:42.809400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.810028 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.810029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:42.810115 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.810120 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:11:42.810177 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 10:11:42.810221 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 10:11:42.810370 20353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.807469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:48843 (local address 127.2.30.129:36309)
0808 10:11:42.807528 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:42.807530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:42.807541 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:11:42.807593 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:42.807594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:42.807610 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:42.807691 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:42.807903 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.808465 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.809498 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.809941 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.810001 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.810008 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:42.810198 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 10:11:42.810244 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 10:11:42.810456 20366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.807449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:40401 (local address 127.2.30.130:57945)
0808 10:11:42.807659 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:42.807671 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:42.807735 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:42.807892 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:42.807893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:42.807894 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:42.807965 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:42.807967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.808732 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.808733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:42.809535 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:42.809538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.810176 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.810177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:42.810251 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.810255 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:11:42.810314 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 10:11:42.810353 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":134,"threads_started":1}
I20260808 10:11:42.810464 20356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.807571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:57945 (local address 127.2.30.131:40401)
0808 10:11:42.807624 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:42.807626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:42.807748 (+ 122us) server_negotiation.cc:407] Connection header received
0808 10:11:42.807801 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:42.807803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:42.807813 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:42.807871 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:42.808035 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.808645 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.809625 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.810062 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.810136 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.810145 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:42.810340 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 10:11:42.810384 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 10:11:42.810922 20005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4edd35174e148a1a4946d52adebfff9" candidate_uuid: "7e5dee9311694229ac852a082eb5b523" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75baaa85e564c8c860d25bdb13e0432" is_pre_election: true
I20260808 10:11:42.810917 20291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4edd35174e148a1a4946d52adebfff9" candidate_uuid: "7e5dee9311694229ac852a082eb5b523" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227a12e2726444f6957ec87f8d1c9ce1" is_pre_election: true
I20260808 10:11:42.811103 20291 raft_consensus.cc:2454] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e5dee9311694229ac852a082eb5b523 in term 0.
I20260808 10:11:42.811116 20005 raft_consensus.cc:2454] T d4edd35174e148a1a4946d52adebfff9 P c75baaa85e564c8c860d25bdb13e0432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e5dee9311694229ac852a082eb5b523 in term 0.
I20260808 10:11:42.811406 20080 leader_election.cc:302] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e5dee9311694229ac852a082eb5b523, c75baaa85e564c8c860d25bdb13e0432; no voters:
I20260808 10:11:42.811533 20365 raft_consensus.cc:2781] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:11:42.811594 20365 raft_consensus.cc:482] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:11:42.811618 20365 raft_consensus.cc:3037] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:42.812605 20365 raft_consensus.cc:504] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75baaa85e564c8c860d25bdb13e0432" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36309 } } peers { permanent_uuid: "227a12e2726444f6957ec87f8d1c9ce1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40401 } } peers { permanent_uuid: "7e5dee9311694229ac852a082eb5b523" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40695 } }
I20260808 10:11:42.812754 20365 leader_election.cc:288] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [CANDIDATE]: Term 1 election: Requested vote from peers c75baaa85e564c8c860d25bdb13e0432 (127.2.30.129:36309), 227a12e2726444f6957ec87f8d1c9ce1 (127.2.30.131:40401)
I20260808 10:11:42.812975 20291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4edd35174e148a1a4946d52adebfff9" candidate_uuid: "7e5dee9311694229ac852a082eb5b523" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227a12e2726444f6957ec87f8d1c9ce1"
I20260808 10:11:42.812954 20005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4edd35174e148a1a4946d52adebfff9" candidate_uuid: "7e5dee9311694229ac852a082eb5b523" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75baaa85e564c8c860d25bdb13e0432"
I20260808 10:11:42.813046 20005 raft_consensus.cc:3037] T d4edd35174e148a1a4946d52adebfff9 P c75baaa85e564c8c860d25bdb13e0432 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:42.813067 20291 raft_consensus.cc:3037] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:42.813761 20291 raft_consensus.cc:2454] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e5dee9311694229ac852a082eb5b523 in term 1.
I20260808 10:11:42.813984 20080 leader_election.cc:302] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227a12e2726444f6957ec87f8d1c9ce1, 7e5dee9311694229ac852a082eb5b523; no voters:
I20260808 10:11:42.813992 20005 raft_consensus.cc:2454] T d4edd35174e148a1a4946d52adebfff9 P c75baaa85e564c8c860d25bdb13e0432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e5dee9311694229ac852a082eb5b523 in term 1.
I20260808 10:11:42.814236 20365 raft_consensus.cc:2781] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:42.814409 20365 raft_consensus.cc:686] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [term 1 LEADER]: Becoming Leader. State: Replica: 7e5dee9311694229ac852a082eb5b523, State: Running, Role: LEADER
I20260808 10:11:42.814503 20365 consensus_queue.cc:237] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75baaa85e564c8c860d25bdb13e0432" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36309 } } peers { permanent_uuid: "227a12e2726444f6957ec87f8d1c9ce1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40401 } } peers { permanent_uuid: "7e5dee9311694229ac852a082eb5b523" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40695 } }
I20260808 10:11:42.815492 19856 catalog_manager.cc:5495] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e5dee9311694229ac852a082eb5b523 (127.2.30.130). New cstate: current_term: 1 leader_uuid: "7e5dee9311694229ac852a082eb5b523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75baaa85e564c8c860d25bdb13e0432" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "227a12e2726444f6957ec87f8d1c9ce1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e5dee9311694229ac852a082eb5b523" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40695 } health_report { overall_health: HEALTHY } } }
I20260808 10:11:42.835865 20355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.833165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39080 (local address 127.2.30.130:40695)
0808 10:11:42.833231 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:42.833233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:42.833243 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:11:42.833321 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:42.833322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:42.833347 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:42.833435 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:11:42.833582 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.834084 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.834861 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:42.835172 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:42.835296 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:42.835490 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:11:42.835608 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:11:42.835704 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:42.835730 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 10:11:42.835750 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 10:11:42.837848 20291 raft_consensus.cc:1261] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1 [term 1 FOLLOWER]: Refusing update from remote peer 7e5dee9311694229ac852a082eb5b523: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:11:42.837870 20005 raft_consensus.cc:1261] T d4edd35174e148a1a4946d52adebfff9 P c75baaa85e564c8c860d25bdb13e0432 [term 1 FOLLOWER]: Refusing update from remote peer 7e5dee9311694229ac852a082eb5b523: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:11:42.838302 20367 consensus_queue.cc:1035] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [LEADER]: Connected to new peer: Peer: permanent_uuid: "227a12e2726444f6957ec87f8d1c9ce1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 40401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:11:42.838442 20365 consensus_queue.cc:1035] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75baaa85e564c8c860d25bdb13e0432" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:11:42.840993 20370 mvcc.cc:205] Tried to move back new op lower bound from 7316209266020757504 to 7316209265929035776. Current Snapshot: MvccSnapshot[applied={T|T < 7316209266020757504}]
I20260808 10:11:42.841588 20372 mvcc.cc:205] Tried to move back new op lower bound from 7316209266020757504 to 7316209265929035776. Current Snapshot: MvccSnapshot[applied={T|T < 7316209266020757504}]
I20260808 10:11:42.844120 19856 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:11:42.844235 19856 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:11:42.845043 19856 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3cbadad8b3e42abbce611a8df82ee97: Sending DeleteTablet for 3 replicas of tablet d4edd35174e148a1a4946d52adebfff9
I20260808 10:11:42.845466 20270 tablet_service.cc:1514] Processing DeleteTablet for tablet d4edd35174e148a1a4946d52adebfff9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:42 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:57738
I20260808 10:11:42.845470 20127 tablet_service.cc:1514] Processing DeleteTablet for tablet d4edd35174e148a1a4946d52adebfff9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:42 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:39076
I20260808 10:11:42.845500 19984 tablet_service.cc:1514] Processing DeleteTablet for tablet d4edd35174e148a1a4946d52adebfff9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:42 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:52258
I20260808 10:11:42.845734 20380 tablet_replica.cc:330] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523: stopping tablet replica
I20260808 10:11:42.845850 20383 tablet_replica.cc:330] T d4edd35174e148a1a4946d52adebfff9 P c75baaa85e564c8c860d25bdb13e0432: stopping tablet replica
I20260808 10:11:42.845856 20380 raft_consensus.cc:2229] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:11:42.845983 20380 raft_consensus.cc:2258] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:42.846040 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 19919
I20260808 10:11:42.848227 20380 ts_tablet_manager.cc:1905] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:11:42.846088 20382 tablet_replica.cc:330] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1: stopping tablet replica
I20260808 10:11:42.848850 20382 raft_consensus.cc:2229] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:11:42.849050 20382 raft_consensus.cc:2258] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:42.849606 20382 ts_tablet_manager.cc:1905] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:11:42.850471 20380 ts_tablet_manager.cc:1918] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:11:42.850607 20380 log.cc:1196] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/d4edd35174e148a1a4946d52adebfff9
I20260808 10:11:42.851033 20380 ts_tablet_manager.cc:1939] T d4edd35174e148a1a4946d52adebfff9 P 7e5dee9311694229ac852a082eb5b523: Deleting consensus metadata
I20260808 10:11:42.851449 20382 ts_tablet_manager.cc:1918] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:11:42.851564 20382 log.cc:1196] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/d4edd35174e148a1a4946d52adebfff9
I20260808 10:11:42.851564 19841 catalog_manager.cc:4841] TS 7e5dee9311694229ac852a082eb5b523 (127.2.30.130:40695): tablet d4edd35174e148a1a4946d52adebfff9 (table test-table [id=62d08be9bfa34122ba0d1e589b65caa1]) successfully deleted
I20260808 10:11:42.851874 20382 ts_tablet_manager.cc:1939] T d4edd35174e148a1a4946d52adebfff9 P 227a12e2726444f6957ec87f8d1c9ce1: Deleting consensus metadata
I20260808 10:11:42.852780 19843 catalog_manager.cc:4841] TS 227a12e2726444f6957ec87f8d1c9ce1 (127.2.30.131:40401): tablet d4edd35174e148a1a4946d52adebfff9 (table test-table [id=62d08be9bfa34122ba0d1e589b65caa1]) successfully deleted
W20260808 10:11:42.854192 19843 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:36309 (error 108)
I20260808 10:11:42.854496 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 20059
I20260808 10:11:42.855301 20351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:42.854610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:36309 (local address 127.0.0.1:52274)
0808 10:11:42.855193 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:42.855259 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:36309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":563}
W20260808 10:11:42.855453 19843 catalog_manager.cc:4584] TS c75baaa85e564c8c860d25bdb13e0432 (127.2.30.129:36309): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d4edd35174e148a1a4946d52adebfff9: Network error: Client connection negotiation failed: client connection to 127.2.30.129:36309: connect: Connection refused (error 111)
I20260808 10:11:42.860766 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 20203
I20260808 10:11:42.866616 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 19823
2026-08-08T10:11:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2125 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T10:11:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:11:42Z Disabled control of system clock
I20260808 10:11:42.900502 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:36117
--webserver_interface=127.2.30.190
--webserver_port=0
--builtin_ntp_servers=127.2.30.148:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:36117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:11:42.986202 20388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:42.986439 20388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:42.986461 20388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:42.987593 20388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:11:42.987630 20388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:42.987643 20388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:42.987658 20388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:11:42.987669 20388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:11:42.988962 20388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:36117
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:36117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:42.989164 20388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:42.989403 20388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:11:42.995303 20388 server_base.cc:1034] running on GCE node
W20260808 10:11:42.995251 20394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:42.995227 20396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:42.995268 20393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:42.995819 20388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:42.996147 20388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:42.997290 20388 hybrid_clock.cc:648] HybridClock initialized: now 1786183902997277 us; error 29 us; skew 500 ppm
I20260808 10:11:42.998651 20388 webserver.cc:460] Webserver started at http://127.2.30.190:36485/ using document root <none> and password file <none>
I20260808 10:11:42.998926 20388 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:42.999033 20388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:42.999205 20388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:43.000376 20388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "ce5ec4aa78234b7285a1a1dc654e974b"
format_stamp: "Formatted at 2026-08-08 10:11:42 on dist-test-slave-w206"
server_key: "41446874f6c5cfa037bf28d73f913bd2"
server_key_iv: "5daa9cc1a5ac78e39ff17ce6ff84e17a"
server_key_version: "encryptionkey@0"
I20260808 10:11:43.000765 20388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "ce5ec4aa78234b7285a1a1dc654e974b"
format_stamp: "Formatted at 2026-08-08 10:11:42 on dist-test-slave-w206"
server_key: "41446874f6c5cfa037bf28d73f913bd2"
server_key_iv: "5daa9cc1a5ac78e39ff17ce6ff84e17a"
server_key_version: "encryptionkey@0"
I20260808 10:11:43.002152 20388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:11:43.002960 20404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:43.003222 20388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:43.003326 20388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "ce5ec4aa78234b7285a1a1dc654e974b"
format_stamp: "Formatted at 2026-08-08 10:11:42 on dist-test-slave-w206"
server_key: "41446874f6c5cfa037bf28d73f913bd2"
server_key_iv: "5daa9cc1a5ac78e39ff17ce6ff84e17a"
server_key_version: "encryptionkey@0"
I20260808 10:11:43.003420 20388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:43.014340 20388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:43.014691 20388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:43.014870 20388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:43.019393 20388 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:36117
I20260808 10:11:43.019539 20458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:36117 every 8 connection(s)
I20260808 10:11:43.019836 20388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:11:43.020859 20459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:43.023255 20459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b: Bootstrap starting.
I20260808 10:11:43.023871 20459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:43.024184 20459 log.cc:824] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:43.024883 20459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b: No bootstrap required, opened a new log
I20260808 10:11:43.025492 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 20388
I20260808 10:11:43.025583 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:11:43.025799 2170 external_mini_cluster.cc:1428] Setting key 6b6e425edcefe58a1d9502fd15bb11f8
I20260808 10:11:43.026907 20459 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5ec4aa78234b7285a1a1dc654e974b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36117 } }
I20260808 10:11:43.027119 20459 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:43.027187 20459 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce5ec4aa78234b7285a1a1dc654e974b, State: Initialized, Role: FOLLOWER
I20260808 10:11:43.027326 20459 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5ec4aa78234b7285a1a1dc654e974b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36117 } }
I20260808 10:11:43.027428 20459 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:11:43.027475 20459 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:11:43.027527 20459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:43.028070 20459 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5ec4aa78234b7285a1a1dc654e974b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36117 } }
I20260808 10:11:43.028205 20459 leader_election.cc:302] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce5ec4aa78234b7285a1a1dc654e974b; no voters:
I20260808 10:11:43.028403 20459 leader_election.cc:288] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:11:43.028448 20465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:43.028617 20465 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [term 1 LEADER]: Becoming Leader. State: Replica: ce5ec4aa78234b7285a1a1dc654e974b, State: Running, Role: LEADER
I20260808 10:11:43.028667 20459 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:11:43.028745 20465 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5ec4aa78234b7285a1a1dc654e974b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36117 } }
I20260808 10:11:43.029137 20465 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce5ec4aa78234b7285a1a1dc654e974b. Latest consensus state: current_term: 1 leader_uuid: "ce5ec4aa78234b7285a1a1dc654e974b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5ec4aa78234b7285a1a1dc654e974b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36117 } } }
I20260808 10:11:43.029255 20465 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:43.029435 20466 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce5ec4aa78234b7285a1a1dc654e974b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5ec4aa78234b7285a1a1dc654e974b" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36117 } } }
I20260808 10:11:43.029501 20466 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:43.029814 20472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:11:43.030344 20472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:11:43.032059 20472 catalog_manager.cc:1347] Generated new cluster ID: 31ca832f34c946b49b3ca462c1a0de98
I20260808 10:11:43.032119 20472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:11:43.032977 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.026374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37672 (local address 127.2.30.190:36117)
0808 10:11:43.026602 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:43.026609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:43.026633 (+ 24us) server_negotiation.cc:407] Connection header received
0808 10:11:43.026906 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:43.026912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:43.027043 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:43.027188 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:43.027432 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.029183 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.030569 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.031248 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.031367 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.032223 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:43.032226 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:43.032227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:43.032455 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:43.032498 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:43.032514 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:43.032585 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:43.032679 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 10:11:43.032763 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":93,"threads_started":1}
I20260808 10:11:43.043187 20472 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:11:43.043741 20472 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:11:43.051107 20472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b: Generated new TSK 0
I20260808 10:11:43.051329 20472 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:11:43.057991 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36117
--builtin_ntp_servers=127.2.30.148:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:11:43.143981 20485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:43.144241 20485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:43.144284 20485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:43.145514 20485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:43.145591 20485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:43.145637 20485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:11:43.147171 20485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:43.147452 20485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:43.147786 20485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:43.153196 20493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:43.153357 20491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:43.153246 20490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:43.153695 20485 server_base.cc:1034] running on GCE node
I20260808 10:11:43.153921 20485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:43.154179 20485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:43.155336 20485 hybrid_clock.cc:648] HybridClock initialized: now 1786183903155295 us; error 55 us; skew 500 ppm
I20260808 10:11:43.156805 20485 webserver.cc:460] Webserver started at http://127.2.30.129:33297/ using document root <none> and password file <none>
I20260808 10:11:43.157086 20485 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:43.157145 20485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:43.157265 20485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:43.158843 20485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "13f92c1190b949d7b90da9c5267e6c6a"
format_stamp: "Formatted at 2026-08-08 10:11:43 on dist-test-slave-w206"
server_key: "f64203bc39a16d495f78f656483b23b0"
server_key_iv: "c70ed14a86cee65edf106f441260a237"
server_key_version: "encryptionkey@0"
I20260808 10:11:43.159313 20485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "13f92c1190b949d7b90da9c5267e6c6a"
format_stamp: "Formatted at 2026-08-08 10:11:43 on dist-test-slave-w206"
server_key: "f64203bc39a16d495f78f656483b23b0"
server_key_iv: "c70ed14a86cee65edf106f441260a237"
server_key_version: "encryptionkey@0"
I20260808 10:11:43.161098 20485 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 10:11:43.162245 20501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:43.162549 20485 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:43.162695 20485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "13f92c1190b949d7b90da9c5267e6c6a"
format_stamp: "Formatted at 2026-08-08 10:11:43 on dist-test-slave-w206"
server_key: "f64203bc39a16d495f78f656483b23b0"
server_key_iv: "c70ed14a86cee65edf106f441260a237"
server_key_version: "encryptionkey@0"
I20260808 10:11:43.162794 20485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:43.185252 20485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:43.185640 20485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:43.185822 20485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:43.186101 20485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:43.186518 20485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:43.186563 20485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:43.186587 20485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:43.186600 20485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:43.192926 20485 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:34947
I20260808 10:11:43.192970 20617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:34947 every 8 connection(s)
I20260808 10:11:43.193419 20485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:11:43.194208 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 20485
I20260808 10:11:43.194334 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:11:43.194581 2170 external_mini_cluster.cc:1428] Setting key dc682996138b47637552dc7c6211099a
I20260808 10:11:43.195755 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36117
--builtin_ntp_servers=127.2.30.148:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:11:43.199925 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.194294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:49637 (local address 127.2.30.190:36117)
0808 10:11:43.194363 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:43.194365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:43.194674 (+ 309us) server_negotiation.cc:407] Connection header received
0808 10:11:43.195342 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:43.195343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:43.195359 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:43.195429 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:43.196406 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.196897 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.198197 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.198547 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.198628 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.199601 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:43.199602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:43.199602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:43.199607 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:43.199624 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:43.199629 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:43.199692 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:43.199820 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 10:11:43.199853 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 10:11:43.200207 20620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.194287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36117 (local address 127.2.30.129:49637)
0808 10:11:43.194509 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:43.194559 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:43.195226 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:43.195530 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:43.195535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:43.195545 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:43.196299 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:43.196306 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.196993 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.196994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:43.198094 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:43.198097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.198641 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.198642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:43.198756 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.199411 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:11:43.199412 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:43.199451 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:11:43.199479 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:43.199706 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:43.199713 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:43.199716 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:11:43.199788 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 10:11:43.199936 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260808 10:11:43.200824 20618 heartbeater.cc:347] Connected to a master server at 127.2.30.190:36117
I20260808 10:11:43.200932 20618 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:43.201133 20618 heartbeater.cc:511] Master 127.2.30.190:36117 requested a full tablet report, sending...
I20260808 10:11:43.201663 20421 ts_manager.cc:194] Registered new tserver with Master: 13f92c1190b949d7b90da9c5267e6c6a (127.2.30.129:34947)
I20260808 10:11:43.202339 20421 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.30.129:49637
W20260808 10:11:43.283372 20621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:43.283603 20621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:43.283627 20621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:43.284852 20621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:43.284901 20621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:43.284927 20621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:11:43.286327 20621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:43.286612 20621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:43.286947 20621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:43.292538 20626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:43.292610 20627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:43.292658 20630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:43.292728 20621 server_base.cc:1034] running on GCE node
I20260808 10:11:43.293097 20621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:43.293330 20621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:43.294476 20621 hybrid_clock.cc:648] HybridClock initialized: now 1786183903294446 us; error 48 us; skew 500 ppm
I20260808 10:11:43.295944 20621 webserver.cc:460] Webserver started at http://127.2.30.130:33167/ using document root <none> and password file <none>
I20260808 10:11:43.296207 20621 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:43.296257 20621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:43.296407 20621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:43.297516 20621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e"
format_stamp: "Formatted at 2026-08-08 10:11:43 on dist-test-slave-w206"
server_key: "3aac9298167612e1579736b83500c11b"
server_key_iv: "b4ee8847500d550c969d817ff09f9ddb"
server_key_version: "encryptionkey@0"
I20260808 10:11:43.297881 20621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e"
format_stamp: "Formatted at 2026-08-08 10:11:43 on dist-test-slave-w206"
server_key: "3aac9298167612e1579736b83500c11b"
server_key_iv: "b4ee8847500d550c969d817ff09f9ddb"
server_key_version: "encryptionkey@0"
I20260808 10:11:43.299328 20621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:11:43.300127 20636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:43.300321 20621 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:43.300428 20621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e"
format_stamp: "Formatted at 2026-08-08 10:11:43 on dist-test-slave-w206"
server_key: "3aac9298167612e1579736b83500c11b"
server_key_iv: "b4ee8847500d550c969d817ff09f9ddb"
server_key_version: "encryptionkey@0"
I20260808 10:11:43.300508 20621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:43.321007 20621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:43.321394 20621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:43.321561 20621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:43.321836 20621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:43.322204 20621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:43.322244 20621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:43.322268 20621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:43.322280 20621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:43.328886 20621 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:40577
I20260808 10:11:43.328938 20754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:40577 every 8 connection(s)
I20260808 10:11:43.329303 20621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:11:43.333457 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 20621
I20260808 10:11:43.333590 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:11:43.333814 2170 external_mini_cluster.cc:1428] Setting key 1086b8b23c5c38cb7dbd1c921f2aeb31
I20260808 10:11:43.335126 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36117
--builtin_ntp_servers=127.2.30.148:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:11:43.335901 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.330365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:35225 (local address 127.2.30.190:36117)
0808 10:11:43.330463 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:43.330466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:43.330608 (+ 142us) server_negotiation.cc:407] Connection header received
0808 10:11:43.331239 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:43.331240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:43.331252 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:43.331302 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:43.332071 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.332581 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.334114 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.334333 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.334420 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.335598 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:43.335599 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:43.335600 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:43.335605 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:43.335620 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:43.335625 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:43.335693 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:43.335779 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 10:11:43.335811 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:11:43.336143 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.330266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36117 (local address 127.2.30.130:35225)
0808 10:11:43.330478 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:43.330518 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:43.331142 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:43.331397 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:43.331401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:43.331410 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:43.331989 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:43.331993 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.332693 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.332693 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:43.334011 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:43.334015 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.334432 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.334434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:43.334622 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.335418 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:11:43.335419 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:43.335459 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:11:43.335492 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:43.335695 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:43.335703 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:43.335706 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:11:43.335774 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 10:11:43.335893 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":70,"threads_started":1}
I20260808 10:11:43.337519 20755 heartbeater.cc:347] Connected to a master server at 127.2.30.190:36117
I20260808 10:11:43.337659 20755 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:43.337885 20755 heartbeater.cc:511] Master 127.2.30.190:36117 requested a full tablet report, sending...
I20260808 10:11:43.338389 20421 ts_manager.cc:194] Registered new tserver with Master: d4cf82fa276a4bf0a79d5478d95b1b9e (127.2.30.130:40577)
I20260808 10:11:43.339006 20421 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.30.130:35225
W20260808 10:11:43.425935 20758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:43.426179 20758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:43.426204 20758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:43.427472 20758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:43.427520 20758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:43.427548 20758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:11:43.429046 20758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:36117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:43.429329 20758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:43.429625 20758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:43.435235 20766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:43.435245 20764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:43.435245 20763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:43.435647 20758 server_base.cc:1034] running on GCE node
I20260808 10:11:43.435937 20758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:43.436211 20758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:43.437402 20758 hybrid_clock.cc:648] HybridClock initialized: now 1786183903437387 us; error 34 us; skew 500 ppm
I20260808 10:11:43.438890 20758 webserver.cc:460] Webserver started at http://127.2.30.131:44415/ using document root <none> and password file <none>
I20260808 10:11:43.439188 20758 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:43.439343 20758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:43.439507 20758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:43.440727 20758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "61a67cae569c4752be9e35f95f7823a1"
format_stamp: "Formatted at 2026-08-08 10:11:43 on dist-test-slave-w206"
server_key: "1db693f8c26466539884e8f443843992"
server_key_iv: "d9832987bac2c05c3f487e4fd440fbc3"
server_key_version: "encryptionkey@0"
I20260808 10:11:43.441125 20758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "61a67cae569c4752be9e35f95f7823a1"
format_stamp: "Formatted at 2026-08-08 10:11:43 on dist-test-slave-w206"
server_key: "1db693f8c26466539884e8f443843992"
server_key_iv: "d9832987bac2c05c3f487e4fd440fbc3"
server_key_version: "encryptionkey@0"
I20260808 10:11:43.442756 20758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:11:43.443796 20772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:43.443985 20758 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:43.444099 20758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "61a67cae569c4752be9e35f95f7823a1"
format_stamp: "Formatted at 2026-08-08 10:11:43 on dist-test-slave-w206"
server_key: "1db693f8c26466539884e8f443843992"
server_key_iv: "d9832987bac2c05c3f487e4fd440fbc3"
server_key_version: "encryptionkey@0"
I20260808 10:11:43.444191 20758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:43.456583 20758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:43.456969 20758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:43.457145 20758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:43.457497 20758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:43.457878 20758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:43.457938 20758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:43.457983 20758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:43.458011 20758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:43.464648 20758 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:39241
I20260808 10:11:43.464690 20890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:39241 every 8 connection(s)
I20260808 10:11:43.465140 20758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:11:43.471541 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.466100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:56079 (local address 127.2.30.190:36117)
0808 10:11:43.466200 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:43.466203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:43.466749 (+ 546us) server_negotiation.cc:407] Connection header received
0808 10:11:43.467416 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:43.467417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:43.467429 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:43.467489 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:43.468266 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.468786 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.469940 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.470189 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.470272 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.471262 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:43.471264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:43.471264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:43.471270 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:43.471285 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:43.471290 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:43.471344 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:43.471440 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 10:11:43.471472 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 10:11:43.471741 20893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.466074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:36117 (local address 127.2.30.131:56079)
0808 10:11:43.466608 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:43.466646 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:43.467301 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:43.467558 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:43.467562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:43.467571 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:43.468125 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:43.468130 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.468892 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.468893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:43.469829 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:43.469832 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.470305 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.470306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:43.470432 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.471096 (+ 664us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:11:43.471097 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:43.471128 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:11:43.471154 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:43.471358 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:43.471364 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:43.471367 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:11:43.471423 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 10:11:43.471517 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":232,"threads_started":1}
I20260808 10:11:43.472494 20891 heartbeater.cc:347] Connected to a master server at 127.2.30.190:36117
I20260808 10:11:43.472491 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 20758
I20260808 10:11:43.472618 20891 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:43.472645 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:11:43.472849 20891 heartbeater.cc:511] Master 127.2.30.190:36117 requested a full tablet report, sending...
I20260808 10:11:43.472863 2170 external_mini_cluster.cc:1428] Setting key 379cb9d2e84e4c79b2aec2de69ae13b8
I20260808 10:11:43.473398 20420 ts_manager.cc:194] Registered new tserver with Master: 61a67cae569c4752be9e35f95f7823a1 (127.2.30.131:39241)
I20260808 10:11:43.473845 20420 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.30.131:56079
I20260808 10:11:43.474583 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:11:43.479518 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.476276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37684 (local address 127.2.30.190:36117)
0808 10:11:43.476434 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:43.476436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:43.476588 (+ 152us) server_negotiation.cc:407] Connection header received
0808 10:11:43.476671 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:43.476672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:43.476686 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:43.476737 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:43.476940 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.477435 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.478204 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.478479 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.478565 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.479256 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:43.479258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:43.479258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:43.479263 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:43.479278 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:43.479282 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:43.479335 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:43.479398 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 10:11:43.479425 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260808 10:11:43.481123 20420 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:11:43.482731 20420 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:11:43.489562 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.485476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:39241 (local address 127.0.0.1:33002)
0808 10:11:43.485619 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:43.485629 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:43.485677 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:43.486008 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:43.486008 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:43.486010 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:43.486043 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:43.486045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.486967 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.486969 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:43.488248 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:43.488251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.489245 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.489246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:43.489326 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.489342 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 10:11:43.489413 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 10:11:43.489470 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260808 10:11:43.489593 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.485605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33002 (local address 127.2.30.131:39241)
0808 10:11:43.485813 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:43.485818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:43.485827 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:11:43.485863 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:43.485865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:43.485877 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:43.485949 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:43.486133 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.486840 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.488342 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.489129 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.489207 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.489304 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:43.489446 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 10:11:43.489494 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":95,"threads_started":1}
I20260808 10:11:43.489792 20899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.485127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:40577 (local address 127.0.0.1:43924)
0808 10:11:43.485328 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:43.485372 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:43.485439 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:43.485908 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:43.485911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:43.485914 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:43.485989 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:43.485991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.487426 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.487427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:43.488457 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:43.488459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.489559 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.489560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:43.489633 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.489637 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:11:43.489670 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 10:11:43.489718 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":89,"threads_started":1}
I20260808 10:11:43.490108 20900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.485235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43924 (local address 127.2.30.130:40577)
0808 10:11:43.485562 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:43.485565 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:43.485576 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:11:43.485627 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:43.485630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:43.485793 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:43.486109 (+ 316us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:43.486180 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.487315 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.488690 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.489459 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.489850 (+ 391us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.489920 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:43.489950 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 10:11:43.490013 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":91,"threads_started":1}
I20260808 10:11:43.490423 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.485235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:34947 (local address 127.0.0.1:49322)
0808 10:11:43.485587 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:43.485597 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:43.485643 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:43.485951 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:43.485952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:43.485953 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:43.486006 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:43.486008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.488363 (+ 2355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.488363 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:43.489109 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:43.489111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.490162 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.490163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:43.490232 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.490236 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:11:43.490291 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 10:11:43.490330 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":120,"threads_started":1}
I20260808 10:11:43.490710 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.485268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49322 (local address 127.2.30.129:34947)
0808 10:11:43.485465 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:43.485471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:43.485708 (+ 237us) server_negotiation.cc:407] Connection header received
0808 10:11:43.485761 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:43.485763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:43.485780 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:43.485854 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:43.486096 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.487324 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.489210 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.489960 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.490035 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.490128 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:43.490574 (+ 446us) server_negotiation.cc:299] Negotiation successful
0808 10:11:43.490627 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":87,"threads_started":1}
I20260808 10:11:43.490916 20824 tablet_service.cc:1467] Processing CreateTablet for tablet 6b7bcebcf63f405c848a36555fd1bdc5 (DEFAULT_TABLE table=test-table [id=99552afc28bd440ca179b8cabdcf437b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:43.491005 20689 tablet_service.cc:1467] Processing CreateTablet for tablet 6b7bcebcf63f405c848a36555fd1bdc5 (DEFAULT_TABLE table=test-table [id=99552afc28bd440ca179b8cabdcf437b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:43.491403 20824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b7bcebcf63f405c848a36555fd1bdc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:43.491418 20689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b7bcebcf63f405c848a36555fd1bdc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:43.491431 20551 tablet_service.cc:1467] Processing CreateTablet for tablet 6b7bcebcf63f405c848a36555fd1bdc5 (DEFAULT_TABLE table=test-table [id=99552afc28bd440ca179b8cabdcf437b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:43.491768 20551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b7bcebcf63f405c848a36555fd1bdc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:43.494814 20905 tablet_bootstrap.cc:492] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e: Bootstrap starting.
I20260808 10:11:43.495066 20906 tablet_bootstrap.cc:492] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1: Bootstrap starting.
I20260808 10:11:43.495455 20907 tablet_bootstrap.cc:492] T 6b7bcebcf63f405c848a36555fd1bdc5 P 13f92c1190b949d7b90da9c5267e6c6a: Bootstrap starting.
I20260808 10:11:43.495695 20905 tablet_bootstrap.cc:654] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:43.495720 20906 tablet_bootstrap.cc:654] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:43.496058 20905 log.cc:824] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:43.496058 20906 log.cc:824] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:43.496106 20907 tablet_bootstrap.cc:654] T 6b7bcebcf63f405c848a36555fd1bdc5 P 13f92c1190b949d7b90da9c5267e6c6a: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:43.496496 20907 log.cc:824] T 6b7bcebcf63f405c848a36555fd1bdc5 P 13f92c1190b949d7b90da9c5267e6c6a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:43.496903 20906 tablet_bootstrap.cc:492] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1: No bootstrap required, opened a new log
I20260808 10:11:43.497026 20906 ts_tablet_manager.cc:1397] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 10:11:43.497084 20905 tablet_bootstrap.cc:492] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e: No bootstrap required, opened a new log
I20260808 10:11:43.497146 20905 ts_tablet_manager.cc:1397] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:11:43.499656 20907 tablet_bootstrap.cc:492] T 6b7bcebcf63f405c848a36555fd1bdc5 P 13f92c1190b949d7b90da9c5267e6c6a: No bootstrap required, opened a new log
I20260808 10:11:43.499792 20907 ts_tablet_manager.cc:1397] T 6b7bcebcf63f405c848a36555fd1bdc5 P 13f92c1190b949d7b90da9c5267e6c6a: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 10:11:43.499822 20906 raft_consensus.cc:348] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40577 } } peers { permanent_uuid: "13f92c1190b949d7b90da9c5267e6c6a" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 34947 } } peers { permanent_uuid: "61a67cae569c4752be9e35f95f7823a1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39241 } }
I20260808 10:11:43.499822 20905 raft_consensus.cc:348] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40577 } } peers { permanent_uuid: "13f92c1190b949d7b90da9c5267e6c6a" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 34947 } } peers { permanent_uuid: "61a67cae569c4752be9e35f95f7823a1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39241 } }
I20260808 10:11:43.500057 20905 raft_consensus.cc:374] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:43.500057 20906 raft_consensus.cc:374] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:43.500108 20906 raft_consensus.cc:729] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61a67cae569c4752be9e35f95f7823a1, State: Initialized, Role: FOLLOWER
I20260808 10:11:43.500137 20905 raft_consensus.cc:729] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4cf82fa276a4bf0a79d5478d95b1b9e, State: Initialized, Role: FOLLOWER
I20260808 10:11:43.500273 20906 consensus_queue.cc:260] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40577 } } peers { permanent_uuid: "13f92c1190b949d7b90da9c5267e6c6a" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 34947 } } peers { permanent_uuid: "61a67cae569c4752be9e35f95f7823a1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39241 } }
I20260808 10:11:43.500299 20905 consensus_queue.cc:260] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40577 } } peers { permanent_uuid: "13f92c1190b949d7b90da9c5267e6c6a" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 34947 } } peers { permanent_uuid: "61a67cae569c4752be9e35f95f7823a1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39241 } }
I20260808 10:11:43.500588 20906 ts_tablet_manager.cc:1428] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260808 10:11:43.500588 20905 ts_tablet_manager.cc:1428] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:11:43.500730 20891 heartbeater.cc:503] Master 127.2.30.190:36117 was elected leader, sending a full tablet report...
I20260808 10:11:43.501065 20755 heartbeater.cc:503] Master 127.2.30.190:36117 was elected leader, sending a full tablet report...
I20260808 10:11:43.502570 20907 raft_consensus.cc:348] T 6b7bcebcf63f405c848a36555fd1bdc5 P 13f92c1190b949d7b90da9c5267e6c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40577 } } peers { permanent_uuid: "13f92c1190b949d7b90da9c5267e6c6a" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 34947 } } peers { permanent_uuid: "61a67cae569c4752be9e35f95f7823a1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39241 } }
I20260808 10:11:43.502703 20907 raft_consensus.cc:374] T 6b7bcebcf63f405c848a36555fd1bdc5 P 13f92c1190b949d7b90da9c5267e6c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:43.502725 20907 raft_consensus.cc:729] T 6b7bcebcf63f405c848a36555fd1bdc5 P 13f92c1190b949d7b90da9c5267e6c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13f92c1190b949d7b90da9c5267e6c6a, State: Initialized, Role: FOLLOWER
I20260808 10:11:43.502856 20907 consensus_queue.cc:260] T 6b7bcebcf63f405c848a36555fd1bdc5 P 13f92c1190b949d7b90da9c5267e6c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40577 } } peers { permanent_uuid: "13f92c1190b949d7b90da9c5267e6c6a" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 34947 } } peers { permanent_uuid: "61a67cae569c4752be9e35f95f7823a1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39241 } }
I20260808 10:11:43.503166 20907 ts_tablet_manager.cc:1428] T 6b7bcebcf63f405c848a36555fd1bdc5 P 13f92c1190b949d7b90da9c5267e6c6a: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260808 10:11:43.503262 20618 heartbeater.cc:503] Master 127.2.30.190:36117 was elected leader, sending a full tablet report...
I20260808 10:11:43.570343 20911 raft_consensus.cc:482] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:11:43.570523 20911 raft_consensus.cc:504] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40577 } } peers { permanent_uuid: "13f92c1190b949d7b90da9c5267e6c6a" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 34947 } } peers { permanent_uuid: "61a67cae569c4752be9e35f95f7823a1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39241 } }
I20260808 10:11:43.570945 20911 leader_election.cc:288] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13f92c1190b949d7b90da9c5267e6c6a (127.2.30.129:34947), 61a67cae569c4752be9e35f95f7823a1 (127.2.30.131:39241)
I20260808 10:11:43.574477 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.571707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:50829 (local address 127.2.30.131:39241)
0808 10:11:43.571796 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:43.571798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:43.571807 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:11:43.571836 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:43.571837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:43.571846 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:43.571913 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:43.572090 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.572582 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.573617 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.573974 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.574164 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.574172 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:43.574344 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 10:11:43.574398 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:11:43.574479 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.571280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:34947 (local address 127.2.30.130:34065)
0808 10:11:43.571443 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:43.571455 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:43.571648 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:43.571814 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:43.571815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:43.571818 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:43.571899 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:43.571902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.572645 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.572646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:43.573481 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:43.573483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.574146 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.574147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:43.574232 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.574235 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:11:43.574280 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 10:11:43.574345 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":5}
I20260808 10:11:43.574488 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.571354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:34065 (local address 127.2.30.129:34947)
0808 10:11:43.571432 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:43.571435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:43.571664 (+ 229us) server_negotiation.cc:407] Connection header received
0808 10:11:43.571721 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:43.571723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:43.571737 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:43.571791 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:43.571996 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.572551 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.573578 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.573967 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.574025 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.574032 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:43.574371 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 10:11:43.574420 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 10:11:43.574488 20914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.571505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:39241 (local address 127.2.30.130:50829)
0808 10:11:43.571669 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:43.571680 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:43.571731 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:43.571913 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:43.571914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:43.571915 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:43.572016 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:43.572018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.572696 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.572697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:43.573490 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:43.573493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.574178 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.574178 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:43.574245 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.574249 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:11:43.574322 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:11:43.574365 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260808 10:11:43.575037 20844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b7bcebcf63f405c848a36555fd1bdc5" candidate_uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61a67cae569c4752be9e35f95f7823a1" is_pre_election: true
I20260808 10:11:43.575038 20571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b7bcebcf63f405c848a36555fd1bdc5" candidate_uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13f92c1190b949d7b90da9c5267e6c6a" is_pre_election: true
I20260808 10:11:43.575217 20844 raft_consensus.cc:2454] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4cf82fa276a4bf0a79d5478d95b1b9e in term 0.
I20260808 10:11:43.575213 20571 raft_consensus.cc:2454] T 6b7bcebcf63f405c848a36555fd1bdc5 P 13f92c1190b949d7b90da9c5267e6c6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4cf82fa276a4bf0a79d5478d95b1b9e in term 0.
I20260808 10:11:43.575484 20638 leader_election.cc:302] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61a67cae569c4752be9e35f95f7823a1, d4cf82fa276a4bf0a79d5478d95b1b9e; no voters:
I20260808 10:11:43.575642 20911 raft_consensus.cc:2781] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:11:43.575721 20911 raft_consensus.cc:482] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:11:43.575740 20911 raft_consensus.cc:3037] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:43.576620 20911 raft_consensus.cc:504] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40577 } } peers { permanent_uuid: "13f92c1190b949d7b90da9c5267e6c6a" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 34947 } } peers { permanent_uuid: "61a67cae569c4752be9e35f95f7823a1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39241 } }
I20260808 10:11:43.576788 20911 leader_election.cc:288] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [CANDIDATE]: Term 1 election: Requested vote from peers 13f92c1190b949d7b90da9c5267e6c6a (127.2.30.129:34947), 61a67cae569c4752be9e35f95f7823a1 (127.2.30.131:39241)
I20260808 10:11:43.576992 20844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b7bcebcf63f405c848a36555fd1bdc5" candidate_uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61a67cae569c4752be9e35f95f7823a1"
I20260808 10:11:43.576983 20571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b7bcebcf63f405c848a36555fd1bdc5" candidate_uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13f92c1190b949d7b90da9c5267e6c6a"
I20260808 10:11:43.577060 20844 raft_consensus.cc:3037] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:43.577085 20571 raft_consensus.cc:3037] T 6b7bcebcf63f405c848a36555fd1bdc5 P 13f92c1190b949d7b90da9c5267e6c6a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:43.577821 20844 raft_consensus.cc:2454] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4cf82fa276a4bf0a79d5478d95b1b9e in term 1.
I20260808 10:11:43.577965 20571 raft_consensus.cc:2454] T 6b7bcebcf63f405c848a36555fd1bdc5 P 13f92c1190b949d7b90da9c5267e6c6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4cf82fa276a4bf0a79d5478d95b1b9e in term 1.
I20260808 10:11:43.578011 20638 leader_election.cc:302] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61a67cae569c4752be9e35f95f7823a1, d4cf82fa276a4bf0a79d5478d95b1b9e; no voters:
I20260808 10:11:43.578119 20911 raft_consensus.cc:2781] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:43.578306 20911 raft_consensus.cc:686] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [term 1 LEADER]: Becoming Leader. State: Replica: d4cf82fa276a4bf0a79d5478d95b1b9e, State: Running, Role: LEADER
I20260808 10:11:43.578408 20911 consensus_queue.cc:237] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40577 } } peers { permanent_uuid: "13f92c1190b949d7b90da9c5267e6c6a" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 34947 } } peers { permanent_uuid: "61a67cae569c4752be9e35f95f7823a1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39241 } }
I20260808 10:11:43.579303 20420 catalog_manager.cc:5495] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e reported cstate change: term changed from 0 to 1, leader changed from <none> to d4cf82fa276a4bf0a79d5478d95b1b9e (127.2.30.130). New cstate: current_term: 1 leader_uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4cf82fa276a4bf0a79d5478d95b1b9e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13f92c1190b949d7b90da9c5267e6c6a" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 34947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61a67cae569c4752be9e35f95f7823a1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39241 } health_report { overall_health: UNKNOWN } } }
W20260808 10:11:43.579862 20756 tablet.cc:2369] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:11:43.607795 20900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.604976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43932 (local address 127.2.30.130:40577)
0808 10:11:43.605044 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:43.605046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:43.605058 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:11:43.605123 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:43.605124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:43.605148 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:43.605284 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:11:43.605428 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.605980 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.606835 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:43.607104 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:43.607179 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:43.607374 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:11:43.607530 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:11:43.607622 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:43.607640 (+ 18us) server_negotiation.cc:299] Negotiation successful
0808 10:11:43.607666 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:11:43.609890 20844 raft_consensus.cc:1261] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1 [term 1 FOLLOWER]: Refusing update from remote peer d4cf82fa276a4bf0a79d5478d95b1b9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:11:43.609978 20571 raft_consensus.cc:1261] T 6b7bcebcf63f405c848a36555fd1bdc5 P 13f92c1190b949d7b90da9c5267e6c6a [term 1 FOLLOWER]: Refusing update from remote peer d4cf82fa276a4bf0a79d5478d95b1b9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:11:43.610325 20911 consensus_queue.cc:1035] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "61a67cae569c4752be9e35f95f7823a1" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 39241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:11:43.610531 20915 consensus_queue.cc:1035] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13f92c1190b949d7b90da9c5267e6c6a" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 34947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:11:43.613684 20920 mvcc.cc:205] Tried to move back new op lower bound from 7316209269182685184 to 7316209269057875968. Current Snapshot: MvccSnapshot[applied={T|T < 7316209269182685184}]
I20260808 10:11:43.613937 20922 mvcc.cc:205] Tried to move back new op lower bound from 7316209269182685184 to 7316209269057875968. Current Snapshot: MvccSnapshot[applied={T|T < 7316209269182685184}]
I20260808 10:11:43.616222 20421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:11:43.616389 20421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:11:43.617419 20421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce5ec4aa78234b7285a1a1dc654e974b: Sending DeleteTablet for 3 replicas of tablet 6b7bcebcf63f405c848a36555fd1bdc5
I20260808 10:11:43.617756 20689 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b7bcebcf63f405c848a36555fd1bdc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:43 UTC) from {username='slave'} at 127.0.0.1:43924
I20260808 10:11:43.617856 20551 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b7bcebcf63f405c848a36555fd1bdc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:43 UTC) from {username='slave'} at 127.0.0.1:49322
I20260808 10:11:43.617858 20824 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b7bcebcf63f405c848a36555fd1bdc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:43 UTC) from {username='slave'} at 127.0.0.1:33002
I20260808 10:11:43.618119 20932 tablet_replica.cc:330] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e: stopping tablet replica
I20260808 10:11:43.618193 20933 tablet_replica.cc:330] T 6b7bcebcf63f405c848a36555fd1bdc5 P 13f92c1190b949d7b90da9c5267e6c6a: stopping tablet replica
I20260808 10:11:43.618373 20933 raft_consensus.cc:2229] T 6b7bcebcf63f405c848a36555fd1bdc5 P 13f92c1190b949d7b90da9c5267e6c6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:11:43.618369 20932 raft_consensus.cc:2229] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:11:43.618464 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 20485
I20260808 10:11:43.618490 20932 raft_consensus.cc:2258] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:43.620069 20932 ts_tablet_manager.cc:1905] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:11:43.622051 20932 ts_tablet_manager.cc:1918] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:11:43.622225 20932 log.cc:1196] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/6b7bcebcf63f405c848a36555fd1bdc5
I20260808 10:11:43.622649 20932 ts_tablet_manager.cc:1939] T 6b7bcebcf63f405c848a36555fd1bdc5 P d4cf82fa276a4bf0a79d5478d95b1b9e: Deleting consensus metadata
I20260808 10:11:43.623216 20407 catalog_manager.cc:4841] TS d4cf82fa276a4bf0a79d5478d95b1b9e (127.2.30.130:40577): tablet 6b7bcebcf63f405c848a36555fd1bdc5 (table test-table [id=99552afc28bd440ca179b8cabdcf437b]) successfully deleted
W20260808 10:11:43.626892 20405 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:34947 (error 108)
I20260808 10:11:43.627009 20934 tablet_replica.cc:330] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1: stopping tablet replica
I20260808 10:11:43.627184 20934 raft_consensus.cc:2229] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:11:43.627338 20934 raft_consensus.cc:2258] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:43.627820 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 20621
I20260808 10:11:43.627849 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:43.627676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:34947 (local address 127.0.0.1:49332)
0808 10:11:43.627747 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:43.627807 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:34947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 10:11:43.628999 20405 catalog_manager.cc:4584] TS 13f92c1190b949d7b90da9c5267e6c6a (127.2.30.129:34947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b7bcebcf63f405c848a36555fd1bdc5: Network error: Client connection negotiation failed: client connection to 127.2.30.129:34947: connect: Connection refused (error 111)
I20260808 10:11:43.631560 20934 ts_tablet_manager.cc:1905] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:11:43.633332 20934 ts_tablet_manager.cc:1918] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:11:43.633401 20934 log.cc:1196] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/6b7bcebcf63f405c848a36555fd1bdc5
I20260808 10:11:43.633610 20934 ts_tablet_manager.cc:1939] T 6b7bcebcf63f405c848a36555fd1bdc5 P 61a67cae569c4752be9e35f95f7823a1: Deleting consensus metadata
I20260808 10:11:43.634116 20405 catalog_manager.cc:4841] TS 61a67cae569c4752be9e35f95f7823a1 (127.2.30.131:39241): tablet 6b7bcebcf63f405c848a36555fd1bdc5 (table test-table [id=99552afc28bd440ca179b8cabdcf437b]) successfully deleted
I20260808 10:11:43.635851 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 20758
I20260808 10:11:43.642354 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 20388
2026-08-08T10:11:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (773 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:11:43 dist-test-slave-w206 krb5kdc[20938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:11:43 dist-test-slave-w206 krb5kdc[20938](info): set up 2 sockets
Aug 08 10:11:43 dist-test-slave-w206 krb5kdc[20938](info): commencing operation
krb5kdc: starting...
W20260808 10:11:44.739794 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:11:44 dist-test-slave-w206 krb5kdc[20938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:11:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:11:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:11:44.878351 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:43959
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:43959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:44.966192 20952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:44.966463 20952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:44.966509 20952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:44.967794 20952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:11:44.967870 20952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:44.967896 20952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:44.967926 20952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:11:44.967958 20952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:11:44.969574 20952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:43959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:43959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:44.969892 20952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:44.970155 20952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:44.975399 20957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:44.975399 20960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:44.975407 20958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:44.975745 20952 server_base.cc:1034] running on GCE node
I20260808 10:11:44.976006 20952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:44.976333 20952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:44.977483 20952 hybrid_clock.cc:648] HybridClock initialized: now 1786183904977438 us; error 63 us; skew 500 ppm
Aug 08 10:11:44 dist-test-slave-w206 krb5kdc[20938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183904, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:44.980458 20952 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:11:44.980937 20952 webserver.cc:460] Webserver started at http://127.2.30.190:35767/ using document root <none> and password file <none>
I20260808 10:11:44.981176 20952 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:44.981222 20952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:44.981341 20952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:44.982491 20952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "012ee85cd380410c8c2858117db39f95"
format_stamp: "Formatted at 2026-08-08 10:11:44 on dist-test-slave-w206"
server_key: "2ad8ddec14ebd039fc8065c742643cb2"
server_key_iv: "033e4e76ee729a29a5fbfab61a9ee9de"
server_key_version: "encryptionkey@0"
I20260808 10:11:44.982831 20952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "012ee85cd380410c8c2858117db39f95"
format_stamp: "Formatted at 2026-08-08 10:11:44 on dist-test-slave-w206"
server_key: "2ad8ddec14ebd039fc8065c742643cb2"
server_key_iv: "033e4e76ee729a29a5fbfab61a9ee9de"
server_key_version: "encryptionkey@0"
I20260808 10:11:44.984431 20952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:44.985256 20967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:44.985492 20952 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:44.985591 20952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "012ee85cd380410c8c2858117db39f95"
format_stamp: "Formatted at 2026-08-08 10:11:44 on dist-test-slave-w206"
server_key: "2ad8ddec14ebd039fc8065c742643cb2"
server_key_iv: "033e4e76ee729a29a5fbfab61a9ee9de"
server_key_version: "encryptionkey@0"
I20260808 10:11:44.985675 20952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:45.022264 20952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:45.024785 20952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:45.024911 20952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:45.029364 20952 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:43959
I20260808 10:11:45.029428 21021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:43959 every 8 connection(s)
I20260808 10:11:45.029788 20952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:11:45.030884 21023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:45.033623 21023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95: Bootstrap starting.
I20260808 10:11:45.034333 21023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:45.034650 21023 log.cc:824] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:45.035259 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 20952
I20260808 10:11:45.035403 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:11:45.035635 2170 external_mini_cluster.cc:1428] Setting key 00f2f7c63ec1fa13d6aa4fed684e1698
I20260808 10:11:45.035646 21023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95: No bootstrap required, opened a new log
I20260808 10:11:45.038369 21023 raft_consensus.cc:348] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "012ee85cd380410c8c2858117db39f95" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43959 } }
I20260808 10:11:45.038533 21023 raft_consensus.cc:374] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:45.038559 21023 raft_consensus.cc:729] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012ee85cd380410c8c2858117db39f95, State: Initialized, Role: FOLLOWER
I20260808 10:11:45.038692 21023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "012ee85cd380410c8c2858117db39f95" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43959 } }
I20260808 10:11:45.038771 21023 raft_consensus.cc:388] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:11:45.038833 21023 raft_consensus.cc:482] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:11:45.038885 21023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:45.039511 21023 raft_consensus.cc:504] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "012ee85cd380410c8c2858117db39f95" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43959 } }
I20260808 10:11:45.039603 21023 leader_election.cc:302] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 012ee85cd380410c8c2858117db39f95; no voters:
I20260808 10:11:45.039817 21023 leader_election.cc:288] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:11:45.039862 21028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:45.040062 21028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [term 1 LEADER]: Becoming Leader. State: Replica: 012ee85cd380410c8c2858117db39f95, State: Running, Role: LEADER
I20260808 10:11:45.040100 21023 sys_catalog.cc:564] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:11:45.040189 21028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "012ee85cd380410c8c2858117db39f95" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43959 } }
I20260808 10:11:45.040517 21028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 012ee85cd380410c8c2858117db39f95. Latest consensus state: current_term: 1 leader_uuid: "012ee85cd380410c8c2858117db39f95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "012ee85cd380410c8c2858117db39f95" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43959 } } }
I20260808 10:11:45.040643 21028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:45.040529 21030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "012ee85cd380410c8c2858117db39f95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "012ee85cd380410c8c2858117db39f95" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 43959 } } }
I20260808 10:11:45.040728 21030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95 [sys.catalog]: This master's current role is: LEADER
W20260808 10:11:45.042125 21044 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:11:45.042270 21044 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:11:45.042382 21035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:11:45.042630 21035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 10:11:45 dist-test-slave-w206 krb5kdc[20938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:11:45.044836 21035 catalog_manager.cc:1347] Generated new cluster ID: 4a75fd1ba50944a39a068b2dccde848c
I20260808 10:11:45.044906 21035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:11:45.048763 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.036268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52378 (local address 127.2.30.190:43959)
0808 10:11:45.036457 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:45.036465 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:45.036490 (+ 25us) server_negotiation.cc:407] Connection header received
0808 10:11:45.037096 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:45.037103 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:45.037228 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:45.037349 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:45.038234 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.039710 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.040898 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.041719 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.041829 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.044822 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:45.044826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:45.044827 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:45.044839 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:45.046946 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:45.047377 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:45.047380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:45.047381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:45.047501 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:45.047729 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:45.047729 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:45.047730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:45.048070 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:45.048256 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:45.048342 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 10:11:45.048498 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260808 10:11:45.064488 21035 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:11:45.065267 21035 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:11:45.073690 21035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95: Generated new TSK 0
I20260808 10:11:45.073936 21035 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:11:45.104627 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43959
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:45.193804 21049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:45.194065 21049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:45.194109 21049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:45.195320 21049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:45.195391 21049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:45.195438 21049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:11:45.196950 21049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:45.197235 21049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:45.197484 21049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:45.203090 21055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:45.203155 21054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:45.203128 21057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:45.203397 21049 server_base.cc:1034] running on GCE node
I20260808 10:11:45.203603 21049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:45.203877 21049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:45.205104 21049 hybrid_clock.cc:648] HybridClock initialized: now 1786183905205103 us; error 43 us; skew 500 ppm
Aug 08 10:11:45 dist-test-slave-w206 krb5kdc[20938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183905, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:45.208402 21049 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:11:45.208998 21049 webserver.cc:460] Webserver started at http://127.2.30.129:35153/ using document root <none> and password file <none>
I20260808 10:11:45.209251 21049 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:45.209327 21049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:45.209486 21049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:45.210649 21049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "144232f5406b45f0aab96770213a956d"
format_stamp: "Formatted at 2026-08-08 10:11:45 on dist-test-slave-w206"
server_key: "3b81066db62586b1331665ced904f5ca"
server_key_iv: "7b5cafa1e57b849e198721fd1d161003"
server_key_version: "encryptionkey@0"
I20260808 10:11:45.211107 21049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "144232f5406b45f0aab96770213a956d"
format_stamp: "Formatted at 2026-08-08 10:11:45 on dist-test-slave-w206"
server_key: "3b81066db62586b1331665ced904f5ca"
server_key_iv: "7b5cafa1e57b849e198721fd1d161003"
server_key_version: "encryptionkey@0"
I20260808 10:11:45.212721 21049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:45.213745 21065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:45.214012 21049 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:45.214155 21049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "144232f5406b45f0aab96770213a956d"
format_stamp: "Formatted at 2026-08-08 10:11:45 on dist-test-slave-w206"
server_key: "3b81066db62586b1331665ced904f5ca"
server_key_iv: "7b5cafa1e57b849e198721fd1d161003"
server_key_version: "encryptionkey@0"
I20260808 10:11:45.214262 21049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:45.224479 21049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:45.227130 21049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:45.227350 21049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:45.227685 21049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:45.228034 21049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:45.228087 21049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:45.228130 21049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:45.228173 21049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:45.234642 21049 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:46375
I20260808 10:11:45.234696 21182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:46375 every 8 connection(s)
I20260808 10:11:45.235088 21049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:11:45.241544 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 21049
I20260808 10:11:45.241662 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:11:45.241878 2170 external_mini_cluster.cc:1428] Setting key 11ab2c479c0fac9b193c4fe4f32edfe0
Aug 08 10:11:45 dist-test-slave-w206 krb5kdc[20938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183905, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:11:45.245814 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.235840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:53433 (local address 127.2.30.190:43959)
0808 10:11:45.235901 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:45.235903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:45.236101 (+ 198us) server_negotiation.cc:407] Connection header received
0808 10:11:45.236643 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:45.236644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:45.236658 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:45.236706 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:45.237716 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.238251 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.239456 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.239686 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.239769 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.242517 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:45.242519 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:45.242519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:45.242529 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:45.244082 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:45.244623 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:45.244625 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:45.244626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:45.244706 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:45.245045 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:45.245046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:45.245046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:45.245172 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:45.245303 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:45.245632 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 10:11:45.245729 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 10:11:45.246152 21185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.235819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:43959 (local address 127.2.30.129:53433)
0808 10:11:45.235979 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:45.236016 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:45.236530 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:45.236795 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:45.236798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:45.237384 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:45.237601 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:45.237605 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.238392 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.238393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:45.239342 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:45.239346 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.239781 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.239782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:45.239940 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.240595 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:11:45.240604 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:45.242418 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:45.244202 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:45.244213 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:45.244216 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:45.244526 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:45.244801 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:45.244802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:45.244802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:45.244967 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:45.245244 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:45.245252 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:45.245442 (+ 190us) client_negotiation.cc:769] Sending connection context
0808 10:11:45.245580 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 10:11:45.245809 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260808 10:11:45.246932 21183 heartbeater.cc:347] Connected to a master server at 127.2.30.190:43959
I20260808 10:11:45.247112 21183 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:45.247350 21183 heartbeater.cc:511] Master 127.2.30.190:43959 requested a full tablet report, sending...
I20260808 10:11:45.247960 20983 ts_manager.cc:194] Registered new tserver with Master: 144232f5406b45f0aab96770213a956d (127.2.30.129:46375)
I20260808 10:11:45.248632 20983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:53433
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:11:45.275450 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43959
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:45.361737 21188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:45.361958 21188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:45.361982 21188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:45.363140 21188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:45.363184 21188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:45.363212 21188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:11:45.364701 21188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:45.364899 21188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:45.365096 21188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:45.370263 21196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:45.370229 21194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:45.370229 21193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:45.370528 21188 server_base.cc:1034] running on GCE node
I20260808 10:11:45.370798 21188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:45.371140 21188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:45.372305 21188 hybrid_clock.cc:648] HybridClock initialized: now 1786183905372303 us; error 39 us; skew 500 ppm
Aug 08 10:11:45 dist-test-slave-w206 krb5kdc[20938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183905, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:45.375063 21188 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:11:45.375521 21188 webserver.cc:460] Webserver started at http://127.2.30.130:41155/ using document root <none> and password file <none>
I20260808 10:11:45.375741 21188 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:45.375784 21188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:45.375880 21188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:45.376937 21188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "0c071c86866e4ea0bd7b0c25ea957f4e"
format_stamp: "Formatted at 2026-08-08 10:11:45 on dist-test-slave-w206"
server_key: "070d2395bbea5c81061278ed5d32b595"
server_key_iv: "0dd586c75c3e688e080000417ee8e218"
server_key_version: "encryptionkey@0"
I20260808 10:11:45.377259 21188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "0c071c86866e4ea0bd7b0c25ea957f4e"
format_stamp: "Formatted at 2026-08-08 10:11:45 on dist-test-slave-w206"
server_key: "070d2395bbea5c81061278ed5d32b595"
server_key_iv: "0dd586c75c3e688e080000417ee8e218"
server_key_version: "encryptionkey@0"
I20260808 10:11:45.378621 21188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:11:45.379428 21204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:45.379575 21188 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:45.379652 21188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "0c071c86866e4ea0bd7b0c25ea957f4e"
format_stamp: "Formatted at 2026-08-08 10:11:45 on dist-test-slave-w206"
server_key: "070d2395bbea5c81061278ed5d32b595"
server_key_iv: "0dd586c75c3e688e080000417ee8e218"
server_key_version: "encryptionkey@0"
I20260808 10:11:45.379777 21188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:45.405133 21188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:45.407485 21188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:45.407644 21188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:45.407927 21188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:45.408279 21188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:45.408345 21188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:45.408394 21188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:45.408427 21188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:45.414494 21188 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:41233
I20260808 10:11:45.414541 21322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:41233 every 8 connection(s)
I20260808 10:11:45.414877 21188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
Aug 08 10:11:45 dist-test-slave-w206 krb5kdc[20938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183905, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:11:45.423233 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 21188
I20260808 10:11:45.423362 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:11:45.423583 2170 external_mini_cluster.cc:1428] Setting key 2d2709bf91c076ab2c3852c777189fbf
I20260808 10:11:45.426548 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.415889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:57509 (local address 127.2.30.190:43959)
0808 10:11:45.416044 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:45.416047 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:45.416276 (+ 229us) server_negotiation.cc:407] Connection header received
0808 10:11:45.417165 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:45.417167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:45.417182 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:45.417235 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:45.418403 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.419106 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.420258 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.420492 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.420574 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.422938 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:45.422939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:45.422940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:45.422947 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:45.424775 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:45.425369 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:45.425370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:45.425370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:45.425431 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:45.425738 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:45.425739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:45.425739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:45.425853 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:45.425911 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:45.426313 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 10:11:45.426415 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260808 10:11:45.426738 21325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.415714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:43959 (local address 127.2.30.130:57509)
0808 10:11:45.416081 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:45.416155 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:45.417003 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:45.417315 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:45.417318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:45.418022 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:45.418230 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:45.418235 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.419234 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.419236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:45.420151 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:45.420153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.420582 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.420583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:45.420717 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.421245 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:11:45.421279 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:45.422819 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:45.424862 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:45.424871 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:45.424874 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:45.425250 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:45.425512 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:45.425513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:45.425514 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:45.425670 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:45.425924 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:45.425931 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:45.426113 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 10:11:45.426235 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 10:11:45.426430 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":196,"threads_started":1}
I20260808 10:11:45.427266 21323 heartbeater.cc:347] Connected to a master server at 127.2.30.190:43959
I20260808 10:11:45.427366 21323 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:45.427582 21323 heartbeater.cc:511] Master 127.2.30.190:43959 requested a full tablet report, sending...
I20260808 10:11:45.428072 20983 ts_manager.cc:194] Registered new tserver with Master: 0c071c86866e4ea0bd7b0c25ea957f4e (127.2.30.130:41233)
I20260808 10:11:45.428603 20983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:57509
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:11:45.454568 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43959
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:45.540781 21328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:45.541042 21328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:45.541086 21328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:45.542531 21328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:45.542614 21328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:45.542667 21328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:11:45.544317 21328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:43959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:45.544628 21328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:45.544886 21328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:45.550333 21334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:45.550472 21328 server_base.cc:1034] running on GCE node
W20260808 10:11:45.550362 21333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:45.550333 21337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:45.550881 21328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:45.551174 21328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:45.552349 21328 hybrid_clock.cc:648] HybridClock initialized: now 1786183905552336 us; error 50 us; skew 500 ppm
Aug 08 10:11:45 dist-test-slave-w206 krb5kdc[20938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183905, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:45.555066 21328 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:11:45.555580 21328 webserver.cc:460] Webserver started at http://127.2.30.131:38631/ using document root <none> and password file <none>
I20260808 10:11:45.555840 21328 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:45.555887 21328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:45.556064 21328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:45.557173 21328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "1fd526393153469ca8aaf27d9db39fbb"
format_stamp: "Formatted at 2026-08-08 10:11:45 on dist-test-slave-w206"
server_key: "6c65354c4c30f4bc7862c7466c30023d"
server_key_iv: "ba4d6fd6fa9b61ce2a0a2c2ee3b1d3af"
server_key_version: "encryptionkey@0"
I20260808 10:11:45.557551 21328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "1fd526393153469ca8aaf27d9db39fbb"
format_stamp: "Formatted at 2026-08-08 10:11:45 on dist-test-slave-w206"
server_key: "6c65354c4c30f4bc7862c7466c30023d"
server_key_iv: "ba4d6fd6fa9b61ce2a0a2c2ee3b1d3af"
server_key_version: "encryptionkey@0"
I20260808 10:11:45.559048 21328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:11:45.559870 21344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:45.560091 21328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:45.560185 21328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "1fd526393153469ca8aaf27d9db39fbb"
format_stamp: "Formatted at 2026-08-08 10:11:45 on dist-test-slave-w206"
server_key: "6c65354c4c30f4bc7862c7466c30023d"
server_key_iv: "ba4d6fd6fa9b61ce2a0a2c2ee3b1d3af"
server_key_version: "encryptionkey@0"
I20260808 10:11:45.560278 21328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:45.576110 21328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:45.578152 21328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:45.578331 21328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:45.578590 21328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:45.578923 21328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:45.578981 21328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:45.579038 21328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:45.579085 21328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:45.585534 21328 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:37053
I20260808 10:11:45.585579 21464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:37053 every 8 connection(s)
I20260808 10:11:45.585914 21328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:11:45.590689 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 21328
I20260808 10:11:45.590804 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:11:45.591048 2170 external_mini_cluster.cc:1428] Setting key 464f1f66661ade965248ed6c461a2817
Aug 08 10:11:45 dist-test-slave-w206 krb5kdc[20938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183905, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:11:45.595562 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.586659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:49247 (local address 127.2.30.190:43959)
0808 10:11:45.586729 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:45.586732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:45.586965 (+ 233us) server_negotiation.cc:407] Connection header received
0808 10:11:45.587525 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:45.587526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:45.587537 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:45.587592 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:45.588518 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.589090 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.590070 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.590254 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.590309 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.592969 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:45.592970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:45.592971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:45.592977 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:45.594262 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:45.594640 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:45.594641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:45.594642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:45.594694 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:45.594931 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:45.594931 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:45.594932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:45.595110 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:45.595169 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:45.595430 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 10:11:45.595497 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:11:45.595710 21467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.586634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:43959 (local address 127.2.30.131:49247)
0808 10:11:45.586833 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:45.586871 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:45.587428 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:45.587643 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:45.587647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:45.588229 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:45.588438 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:45.588444 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.589188 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.589189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:45.589995 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:45.589997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.590335 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.590335 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:45.590436 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.591177 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:11:45.591188 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:45.592856 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:45.594344 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:45.594351 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:45.594353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:45.594571 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:45.594756 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:45.594756 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:45.594756 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:45.594873 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:45.595169 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:45.595174 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:45.595295 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 10:11:45.595382 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:11:45.595509 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":121,"threads_started":1}
I20260808 10:11:45.596303 21465 heartbeater.cc:347] Connected to a master server at 127.2.30.190:43959
I20260808 10:11:45.596391 21465 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:45.596597 21465 heartbeater.cc:511] Master 127.2.30.190:43959 requested a full tablet report, sending...
I20260808 10:11:45.596975 20983 ts_manager.cc:194] Registered new tserver with Master: 1fd526393153469ca8aaf27d9db39fbb (127.2.30.131:37053)
I20260808 10:11:45.597375 20983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:49247
I20260808 10:11:45.603174 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:11:45.610786 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.604682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52382 (local address 127.2.30.190:43959)
0808 10:11:45.604731 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:45.604733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:45.604837 (+ 104us) server_negotiation.cc:407] Connection header received
0808 10:11:45.604891 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:45.604892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:45.604901 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:45.604949 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:45.605548 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.606025 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.606743 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.606922 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.607032 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.608438 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:45.608439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:45.608440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:45.608446 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:45.609635 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:45.610011 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:45.610011 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:45.610012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:45.610065 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:45.610285 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:45.610286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:45.610286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:45.610428 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:45.610509 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:45.610612 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 10:11:45.610663 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:11:45.612654 20983 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:11:45.614305 20983 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:11:45.621428 21481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.617220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57166 (local address 127.2.30.129:46375)
0808 10:11:45.617441 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:45.617446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:45.617455 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:11:45.617492 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:45.617494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:45.617510 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:45.617615 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:45.617746 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.618846 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.620490 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.621001 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.621098 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.621162 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:45.621296 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 10:11:45.621336 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":101,"threads_started":1}
I20260808 10:11:45.621441 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.616820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:46375 (local address 127.0.0.1:57166)
0808 10:11:45.617043 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:45.617055 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:45.617112 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:45.617629 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:45.617630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:45.617633 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:45.617679 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:45.617681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.618957 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.618958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:45.620374 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:45.620378 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.621101 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.621101 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:45.621183 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.621197 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 10:11:45.621270 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 10:11:45.621330 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":62,"thread_start_us":110,"threads_started":1}
I20260808 10:11:45.621568 21479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.617046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:41233 (local address 127.0.0.1:39572)
0808 10:11:45.617276 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:45.617284 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:45.617342 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:45.618160 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:45.618161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:45.618162 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:45.618194 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:45.618195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.619202 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.619203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:45.620194 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:45.620197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.621199 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.621200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:45.621282 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.621289 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 10:11:45.621336 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 10:11:45.621380 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":84,"threads_started":1}
I20260808 10:11:45.621625 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.617516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39572 (local address 127.2.30.130:41233)
0808 10:11:45.617873 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:45.617878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:45.617887 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:11:45.617927 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:45.617929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:45.617945 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:45.618079 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:45.618274 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.619097 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.620283 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.621016 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.621100 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.621160 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:45.621485 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 10:11:45.621533 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":109,"threads_started":1}
I20260808 10:11:45.621946 21477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.616791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:37053 (local address 127.0.0.1:42592)
0808 10:11:45.616940 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:45.616983 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:45.617047 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:45.617442 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:45.617445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:45.617447 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:45.617537 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:45.617539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.619004 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.619005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:45.620376 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:45.620378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.621318 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.621319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:45.621381 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.621384 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:11:45.621413 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 10:11:45.621448 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":77,"thread_start_us":67,"threads_started":1}
I20260808 10:11:45.622323 21480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.617022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42592 (local address 127.2.30.131:37053)
0808 10:11:45.617269 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:45.617275 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:45.617286 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:11:45.617332 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:45.617334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:45.617350 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:45.617433 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:45.617626 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.618823 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.620476 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.621224 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.621489 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.622169 (+ 680us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:45.622197 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 10:11:45.622242 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":116,"threads_started":1}
I20260808 10:11:45.622412 21254 tablet_service.cc:1467] Processing CreateTablet for tablet 1dfeee285408455cb3f30b4309bb7e32 (DEFAULT_TABLE table=test-table [id=a8db67703bf54138a39f028f3811747b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:45.622670 21116 tablet_service.cc:1467] Processing CreateTablet for tablet 1dfeee285408455cb3f30b4309bb7e32 (DEFAULT_TABLE table=test-table [id=a8db67703bf54138a39f028f3811747b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:45.622802 21254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dfeee285408455cb3f30b4309bb7e32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:45.622926 21116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dfeee285408455cb3f30b4309bb7e32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:45.622993 21398 tablet_service.cc:1467] Processing CreateTablet for tablet 1dfeee285408455cb3f30b4309bb7e32 (DEFAULT_TABLE table=test-table [id=a8db67703bf54138a39f028f3811747b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:45.623299 21398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dfeee285408455cb3f30b4309bb7e32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:45.626034 21483 tablet_bootstrap.cc:492] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d: Bootstrap starting.
I20260808 10:11:45.626329 21484 tablet_bootstrap.cc:492] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e: Bootstrap starting.
I20260808 10:11:45.626583 21485 tablet_bootstrap.cc:492] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb: Bootstrap starting.
I20260808 10:11:45.626902 21483 tablet_bootstrap.cc:654] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:45.627045 21484 tablet_bootstrap.cc:654] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:45.627312 21485 tablet_bootstrap.cc:654] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:45.627650 21484 log.cc:824] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:45.627650 21483 log.cc:824] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:45.627876 21485 log.cc:824] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:45.628539 21483 tablet_bootstrap.cc:492] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d: No bootstrap required, opened a new log
I20260808 10:11:45.628629 21483 ts_tablet_manager.cc:1397] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:11:45.628683 21484 tablet_bootstrap.cc:492] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e: No bootstrap required, opened a new log
I20260808 10:11:45.628690 21485 tablet_bootstrap.cc:492] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb: No bootstrap required, opened a new log
I20260808 10:11:45.628759 21485 ts_tablet_manager.cc:1397] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 10:11:45.628759 21484 ts_tablet_manager.cc:1397] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 10:11:45.630355 21483 raft_consensus.cc:348] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd526393153469ca8aaf27d9db39fbb" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37053 } } peers { permanent_uuid: "144232f5406b45f0aab96770213a956d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 46375 } } peers { permanent_uuid: "0c071c86866e4ea0bd7b0c25ea957f4e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41233 } }
I20260808 10:11:45.630529 21483 raft_consensus.cc:374] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:45.630577 21483 raft_consensus.cc:729] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144232f5406b45f0aab96770213a956d, State: Initialized, Role: FOLLOWER
I20260808 10:11:45.630692 21483 consensus_queue.cc:260] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd526393153469ca8aaf27d9db39fbb" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37053 } } peers { permanent_uuid: "144232f5406b45f0aab96770213a956d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 46375 } } peers { permanent_uuid: "0c071c86866e4ea0bd7b0c25ea957f4e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41233 } }
I20260808 10:11:45.630869 21485 raft_consensus.cc:348] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd526393153469ca8aaf27d9db39fbb" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37053 } } peers { permanent_uuid: "144232f5406b45f0aab96770213a956d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 46375 } } peers { permanent_uuid: "0c071c86866e4ea0bd7b0c25ea957f4e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41233 } }
I20260808 10:11:45.631017 21183 heartbeater.cc:503] Master 127.2.30.190:43959 was elected leader, sending a full tablet report...
I20260808 10:11:45.630869 21484 raft_consensus.cc:348] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd526393153469ca8aaf27d9db39fbb" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37053 } } peers { permanent_uuid: "144232f5406b45f0aab96770213a956d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 46375 } } peers { permanent_uuid: "0c071c86866e4ea0bd7b0c25ea957f4e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41233 } }
I20260808 10:11:45.631069 21485 raft_consensus.cc:374] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:45.631114 21485 raft_consensus.cc:729] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd526393153469ca8aaf27d9db39fbb, State: Initialized, Role: FOLLOWER
I20260808 10:11:45.631134 21484 raft_consensus.cc:374] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:45.631198 21484 raft_consensus.cc:729] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c071c86866e4ea0bd7b0c25ea957f4e, State: Initialized, Role: FOLLOWER
I20260808 10:11:45.631028 21483 ts_tablet_manager.cc:1428] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 10:11:45.631274 21485 consensus_queue.cc:260] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd526393153469ca8aaf27d9db39fbb" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37053 } } peers { permanent_uuid: "144232f5406b45f0aab96770213a956d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 46375 } } peers { permanent_uuid: "0c071c86866e4ea0bd7b0c25ea957f4e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41233 } }
I20260808 10:11:45.631392 21484 consensus_queue.cc:260] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd526393153469ca8aaf27d9db39fbb" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37053 } } peers { permanent_uuid: "144232f5406b45f0aab96770213a956d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 46375 } } peers { permanent_uuid: "0c071c86866e4ea0bd7b0c25ea957f4e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41233 } }
I20260808 10:11:45.631636 21485 ts_tablet_manager.cc:1428] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:11:45.631757 21323 heartbeater.cc:503] Master 127.2.30.190:43959 was elected leader, sending a full tablet report...
I20260808 10:11:45.631770 21484 ts_tablet_manager.cc:1428] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 10:11:45.631826 21465 heartbeater.cc:503] Master 127.2.30.190:43959 was elected leader, sending a full tablet report...
W20260808 10:11:45.665580 21324 tablet.cc:2369] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:11:45.677076 21491 raft_consensus.cc:482] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:11:45.677253 21491 raft_consensus.cc:504] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd526393153469ca8aaf27d9db39fbb" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37053 } } peers { permanent_uuid: "144232f5406b45f0aab96770213a956d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 46375 } } peers { permanent_uuid: "0c071c86866e4ea0bd7b0c25ea957f4e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41233 } }
I20260808 10:11:45.677663 21491 leader_election.cc:288] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fd526393153469ca8aaf27d9db39fbb (127.2.30.131:37053), 0c071c86866e4ea0bd7b0c25ea957f4e (127.2.30.130:41233)
I20260808 10:11:45.680985 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.677957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:35663 (local address 127.2.30.130:41233)
0808 10:11:45.678006 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:45.678009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:45.678019 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:11:45.678055 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:45.678056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:45.678070 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:45.678147 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:45.678416 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.678931 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.680088 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.680584 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.680641 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.680649 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:45.680857 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 10:11:45.680889 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:11:45.680981 21185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.677862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:41233 (local address 127.2.30.129:35663)
0808 10:11:45.677927 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:45.677946 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:45.677994 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:45.678184 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:45.678186 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:45.678188 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:45.678322 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:45.678325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.679037 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.679038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:45.679995 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:45.679998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.680667 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.680668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:45.680775 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.680780 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:11:45.680847 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 10:11:45.680891 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 10:11:45.681284 21480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.677957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:50685 (local address 127.2.30.131:37053)
0808 10:11:45.678029 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:45.678032 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:45.678201 (+ 169us) server_negotiation.cc:407] Connection header received
0808 10:11:45.678241 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:45.678242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:45.678252 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:45.678316 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:45.678623 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.679300 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.680372 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.680876 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.680940 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.680948 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:45.681149 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 10:11:45.681196 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 10:11:45.681526 21494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.677862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:37053 (local address 127.2.30.129:50685)
0808 10:11:45.678108 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:45.678120 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:45.678183 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:45.678346 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:45.678347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:45.678348 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:45.678442 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:45.678444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.679403 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.679404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:45.680270 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:45.680273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.680966 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.680967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:45.681042 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.681047 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:11:45.681410 (+ 363us) client_negotiation.cc:240] Negotiation successful
0808 10:11:45.681462 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":21,"thread_start_us":149,"threads_started":1}
I20260808 10:11:45.681478 21274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dfeee285408455cb3f30b4309bb7e32" candidate_uuid: "144232f5406b45f0aab96770213a956d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c071c86866e4ea0bd7b0c25ea957f4e" is_pre_election: true
I20260808 10:11:45.681668 21274 raft_consensus.cc:2454] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 144232f5406b45f0aab96770213a956d in term 0.
I20260808 10:11:45.681957 21067 leader_election.cc:302] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c071c86866e4ea0bd7b0c25ea957f4e, 144232f5406b45f0aab96770213a956d; no voters:
I20260808 10:11:45.681946 21419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dfeee285408455cb3f30b4309bb7e32" candidate_uuid: "144232f5406b45f0aab96770213a956d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd526393153469ca8aaf27d9db39fbb" is_pre_election: true
I20260808 10:11:45.682086 21419 raft_consensus.cc:2454] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 144232f5406b45f0aab96770213a956d in term 0.
I20260808 10:11:45.682116 21491 raft_consensus.cc:2781] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:11:45.682228 21491 raft_consensus.cc:482] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:11:45.682263 21491 raft_consensus.cc:3037] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:45.683094 21491 raft_consensus.cc:504] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd526393153469ca8aaf27d9db39fbb" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37053 } } peers { permanent_uuid: "144232f5406b45f0aab96770213a956d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 46375 } } peers { permanent_uuid: "0c071c86866e4ea0bd7b0c25ea957f4e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41233 } }
I20260808 10:11:45.683256 21491 leader_election.cc:288] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [CANDIDATE]: Term 1 election: Requested vote from peers 1fd526393153469ca8aaf27d9db39fbb (127.2.30.131:37053), 0c071c86866e4ea0bd7b0c25ea957f4e (127.2.30.130:41233)
I20260808 10:11:45.683439 21419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dfeee285408455cb3f30b4309bb7e32" candidate_uuid: "144232f5406b45f0aab96770213a956d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd526393153469ca8aaf27d9db39fbb"
I20260808 10:11:45.683507 21419 raft_consensus.cc:3037] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:45.684288 21419 raft_consensus.cc:2454] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144232f5406b45f0aab96770213a956d in term 1.
I20260808 10:11:45.684427 21274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dfeee285408455cb3f30b4309bb7e32" candidate_uuid: "144232f5406b45f0aab96770213a956d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c071c86866e4ea0bd7b0c25ea957f4e"
I20260808 10:11:45.684491 21069 leader_election.cc:302] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144232f5406b45f0aab96770213a956d, 1fd526393153469ca8aaf27d9db39fbb; no voters:
I20260808 10:11:45.684544 21274 raft_consensus.cc:3037] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:45.684592 21491 raft_consensus.cc:2781] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:45.684789 21491 raft_consensus.cc:686] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [term 1 LEADER]: Becoming Leader. State: Replica: 144232f5406b45f0aab96770213a956d, State: Running, Role: LEADER
I20260808 10:11:45.684890 21491 consensus_queue.cc:237] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd526393153469ca8aaf27d9db39fbb" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37053 } } peers { permanent_uuid: "144232f5406b45f0aab96770213a956d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 46375 } } peers { permanent_uuid: "0c071c86866e4ea0bd7b0c25ea957f4e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41233 } }
I20260808 10:11:45.685416 21274 raft_consensus.cc:2454] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144232f5406b45f0aab96770213a956d in term 1.
I20260808 10:11:45.685791 20983 catalog_manager.cc:5495] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d reported cstate change: term changed from 0 to 1, leader changed from <none> to 144232f5406b45f0aab96770213a956d (127.2.30.129). New cstate: current_term: 1 leader_uuid: "144232f5406b45f0aab96770213a956d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd526393153469ca8aaf27d9db39fbb" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144232f5406b45f0aab96770213a956d" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 46375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c071c86866e4ea0bd7b0c25ea957f4e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41233 } health_report { overall_health: UNKNOWN } } }
I20260808 10:11:45.696499 21481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.693815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50376 (local address 127.2.30.129:46375)
0808 10:11:45.693915 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:45.693917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:45.693926 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:11:45.693964 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:45.693965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:45.693985 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:45.694053 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:11:45.694201 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.694723 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.695648 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.695854 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.695923 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.696059 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:11:45.696148 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:11:45.696224 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:45.696386 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 10:11:45.696400 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 10:11:45.698652 21419 raft_consensus.cc:1261] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb [term 1 FOLLOWER]: Refusing update from remote peer 144232f5406b45f0aab96770213a956d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:11:45.698664 21274 raft_consensus.cc:1261] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e [term 1 FOLLOWER]: Refusing update from remote peer 144232f5406b45f0aab96770213a956d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:11:45.699139 21495 consensus_queue.cc:1035] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd526393153469ca8aaf27d9db39fbb" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:11:45.699302 21491 consensus_queue.cc:1035] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c071c86866e4ea0bd7b0c25ea957f4e" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 41233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:11:45.702160 21500 mvcc.cc:205] Tried to move back new op lower bound from 7316209277738450944 to 7316209277685948416. Current Snapshot: MvccSnapshot[applied={T|T < 7316209277738450944}]
I20260808 10:11:45.704449 20984 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:11:45.704568 20984 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:11:45.705327 20984 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 012ee85cd380410c8c2858117db39f95: Sending DeleteTablet for 3 replicas of tablet 1dfeee285408455cb3f30b4309bb7e32
I20260808 10:11:45.705678 21398 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dfeee285408455cb3f30b4309bb7e32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:45 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:42592
I20260808 10:11:45.705752 21116 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dfeee285408455cb3f30b4309bb7e32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:45 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:57166
I20260808 10:11:45.705760 21254 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dfeee285408455cb3f30b4309bb7e32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:45 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:39572
I20260808 10:11:45.705966 21511 tablet_replica.cc:330] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d: stopping tablet replica
I20260808 10:11:45.706092 21511 raft_consensus.cc:2229] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:11:45.706194 21511 raft_consensus.cc:2258] T 1dfeee285408455cb3f30b4309bb7e32 P 144232f5406b45f0aab96770213a956d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:45.706367 21510 tablet_replica.cc:330] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e: stopping tablet replica
I20260808 10:11:45.706396 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 21049
I20260808 10:11:45.706514 21510 raft_consensus.cc:2229] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:11:45.706611 21510 raft_consensus.cc:2258] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:45.707868 21510 ts_tablet_manager.cc:1905] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:11:45.710063 21510 ts_tablet_manager.cc:1918] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:11:45.710143 21510 log.cc:1196] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/1dfeee285408455cb3f30b4309bb7e32
I20260808 10:11:45.710448 21510 ts_tablet_manager.cc:1939] T 1dfeee285408455cb3f30b4309bb7e32 P 0c071c86866e4ea0bd7b0c25ea957f4e: Deleting consensus metadata
I20260808 10:11:45.711294 20969 catalog_manager.cc:4841] TS 0c071c86866e4ea0bd7b0c25ea957f4e (127.2.30.130:41233): tablet 1dfeee285408455cb3f30b4309bb7e32 (table test-table [id=a8db67703bf54138a39f028f3811747b]) successfully deleted
I20260808 10:11:45.715205 21509 tablet_replica.cc:330] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb: stopping tablet replica
I20260808 10:11:45.715409 21509 raft_consensus.cc:2229] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:11:45.715574 21509 raft_consensus.cc:2258] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:45.716063 21509 ts_tablet_manager.cc:1905] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 10:11:45.717203 20969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:46375 (error 108)
I20260808 10:11:45.717388 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 21188
I20260808 10:11:45.718394 21509 ts_tablet_manager.cc:1918] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:11:45.718477 21509 log.cc:1196] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/1dfeee285408455cb3f30b4309bb7e32
I20260808 10:11:45.718817 21509 ts_tablet_manager.cc:1939] T 1dfeee285408455cb3f30b4309bb7e32 P 1fd526393153469ca8aaf27d9db39fbb: Deleting consensus metadata
I20260808 10:11:45.719359 20971 catalog_manager.cc:4841] TS 1fd526393153469ca8aaf27d9db39fbb (127.2.30.131:37053): tablet 1dfeee285408455cb3f30b4309bb7e32 (table test-table [id=a8db67703bf54138a39f028f3811747b]) successfully deleted
I20260808 10:11:45.720912 21477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.720710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:46375 (local address 127.0.0.1:50392)
0808 10:11:45.720791 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:45.720853 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:46375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 10:11:45.721160 20969 catalog_manager.cc:4584] TS 144232f5406b45f0aab96770213a956d (127.2.30.129:46375): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1dfeee285408455cb3f30b4309bb7e32: Network error: Client connection negotiation failed: client connection to 127.2.30.129:46375: connect: Connection refused (error 111)
I20260808 10:11:45.725072 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 21328
I20260808 10:11:45.731496 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 20952
2026-08-08T10:11:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2090 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T10:11:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:11:45Z Disabled control of system clock
I20260808 10:11:45.763913 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:46239
--webserver_interface=127.2.30.190
--webserver_port=0
--builtin_ntp_servers=127.2.30.148:35855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:46239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:11:45.848551 21517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:45.848773 21517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:45.848796 21517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:45.850010 21517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:11:45.850057 21517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:45.850070 21517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:45.850083 21517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:11:45.850147 21517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:11:45.851572 21517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:46239
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:46239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:45.851840 21517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:45.852152 21517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:45.857352 21524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:45.857358 21525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:45.857440 21527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:45.857547 21517 server_base.cc:1034] running on GCE node
I20260808 10:11:45.857988 21517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:45.858345 21517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:45.859519 21517 hybrid_clock.cc:648] HybridClock initialized: now 1786183905859496 us; error 38 us; skew 500 ppm
I20260808 10:11:45.868724 21517 webserver.cc:460] Webserver started at http://127.2.30.190:43019/ using document root <none> and password file <none>
I20260808 10:11:45.869019 21517 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:45.869096 21517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:45.869261 21517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:45.870457 21517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "d2de8c415c4e44e6a01d193fd8902fdf"
format_stamp: "Formatted at 2026-08-08 10:11:45 on dist-test-slave-w206"
server_key: "1924f9b7b2bb9e615cd9817f181ec79d"
server_key_iv: "4b9d6e523b29809dbfc27b63d51366ee"
server_key_version: "encryptionkey@0"
I20260808 10:11:45.870847 21517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "d2de8c415c4e44e6a01d193fd8902fdf"
format_stamp: "Formatted at 2026-08-08 10:11:45 on dist-test-slave-w206"
server_key: "1924f9b7b2bb9e615cd9817f181ec79d"
server_key_iv: "4b9d6e523b29809dbfc27b63d51366ee"
server_key_version: "encryptionkey@0"
I20260808 10:11:45.872283 21517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:45.873144 21533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:45.873453 21517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:45.873569 21517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "d2de8c415c4e44e6a01d193fd8902fdf"
format_stamp: "Formatted at 2026-08-08 10:11:45 on dist-test-slave-w206"
server_key: "1924f9b7b2bb9e615cd9817f181ec79d"
server_key_iv: "4b9d6e523b29809dbfc27b63d51366ee"
server_key_version: "encryptionkey@0"
I20260808 10:11:45.873661 21517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:45.893982 21517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:45.894362 21517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:45.894546 21517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:45.899036 21517 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:46239
I20260808 10:11:45.899125 21589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:46239 every 8 connection(s)
I20260808 10:11:45.899494 21517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:11:45.900674 21590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:45.903239 21590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf: Bootstrap starting.
I20260808 10:11:45.903858 21590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:45.904119 21590 log.cc:824] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:45.904865 21590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf: No bootstrap required, opened a new log
I20260808 10:11:45.906512 21590 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2de8c415c4e44e6a01d193fd8902fdf" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46239 } }
I20260808 10:11:45.906653 21590 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:45.906675 21590 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2de8c415c4e44e6a01d193fd8902fdf, State: Initialized, Role: FOLLOWER
I20260808 10:11:45.906826 21590 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2de8c415c4e44e6a01d193fd8902fdf" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46239 } }
I20260808 10:11:45.906919 21590 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:11:45.906942 21590 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:11:45.907018 21590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:45.907565 21590 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2de8c415c4e44e6a01d193fd8902fdf" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46239 } }
I20260808 10:11:45.907652 21590 leader_election.cc:302] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2de8c415c4e44e6a01d193fd8902fdf; no voters:
I20260808 10:11:45.907806 21590 leader_election.cc:288] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:11:45.907900 21593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:45.908066 21593 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [term 1 LEADER]: Becoming Leader. State: Replica: d2de8c415c4e44e6a01d193fd8902fdf, State: Running, Role: LEADER
I20260808 10:11:45.908083 21590 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:11:45.908190 21593 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2de8c415c4e44e6a01d193fd8902fdf" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46239 } }
I20260808 10:11:45.908740 21595 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2de8c415c4e44e6a01d193fd8902fdf. Latest consensus state: current_term: 1 leader_uuid: "d2de8c415c4e44e6a01d193fd8902fdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2de8c415c4e44e6a01d193fd8902fdf" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46239 } } }
I20260808 10:11:45.908854 21595 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:45.909019 21594 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2de8c415c4e44e6a01d193fd8902fdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2de8c415c4e44e6a01d193fd8902fdf" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46239 } } }
I20260808 10:11:45.909099 21594 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:45.909190 21602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:11:45.909456 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 21517
I20260808 10:11:45.909651 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:11:45.909761 21602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:11:45.909873 2170 external_mini_cluster.cc:1428] Setting key 330ed39d9891b44b76f3ab553234edb7
I20260808 10:11:45.911655 21602 catalog_manager.cc:1347] Generated new cluster ID: 2dc37d5361b04f9cb67a6b60faa8c413
I20260808 10:11:45.911713 21602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:11:45.915696 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:45.910564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47412 (local address 127.2.30.190:46239)
0808 10:11:45.911025 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:45.911034 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:45.911070 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:11:45.911183 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:45.911192 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:45.911262 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:45.911424 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:45.911671 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.912657 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.913570 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:45.914030 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:45.914167 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:45.914950 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:45.914954 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:45.914954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:45.915173 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:45.915207 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:45.915218 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:45.915311 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:45.915388 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 10:11:45.915471 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":92,"threads_started":1}
I20260808 10:11:45.919647 21602 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:11:45.920248 21602 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:11:45.929383 21602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf: Generated new TSK 0
I20260808 10:11:45.929589 21602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:11:45.937709 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46239
--builtin_ntp_servers=127.2.30.148:35855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:11:46.025481 21614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:46.025696 21614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:46.025719 21614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:46.026932 21614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:46.027074 21614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:46.027153 21614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:11:46.028607 21614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:46.028873 21614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:46.029215 21614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:46.035897 21620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:46.035975 21614 server_base.cc:1034] running on GCE node
W20260808 10:11:46.036051 21623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:46.035909 21619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:46.036348 21614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:46.036659 21614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:46.037834 21614 hybrid_clock.cc:648] HybridClock initialized: now 1786183906037812 us; error 44 us; skew 500 ppm
I20260808 10:11:46.039300 21614 webserver.cc:460] Webserver started at http://127.2.30.129:39301/ using document root <none> and password file <none>
I20260808 10:11:46.039587 21614 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:46.039636 21614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:46.039809 21614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:46.041026 21614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "8fe889eb21434fcf9862494251dc7324"
format_stamp: "Formatted at 2026-08-08 10:11:46 on dist-test-slave-w206"
server_key: "7fc2672088a2aaae382d39fe3e97039a"
server_key_iv: "b22de6fe1388d064f4fec7f432370fd9"
server_key_version: "encryptionkey@0"
I20260808 10:11:46.041393 21614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "8fe889eb21434fcf9862494251dc7324"
format_stamp: "Formatted at 2026-08-08 10:11:46 on dist-test-slave-w206"
server_key: "7fc2672088a2aaae382d39fe3e97039a"
server_key_iv: "b22de6fe1388d064f4fec7f432370fd9"
server_key_version: "encryptionkey@0"
I20260808 10:11:46.042780 21614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:11:46.043677 21629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:46.043834 21614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:46.043928 21614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "8fe889eb21434fcf9862494251dc7324"
format_stamp: "Formatted at 2026-08-08 10:11:46 on dist-test-slave-w206"
server_key: "7fc2672088a2aaae382d39fe3e97039a"
server_key_iv: "b22de6fe1388d064f4fec7f432370fd9"
server_key_version: "encryptionkey@0"
I20260808 10:11:46.044006 21614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:46.069573 21614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:46.069934 21614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:46.070093 21614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:46.070349 21614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:46.070715 21614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:46.070778 21614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:46.070823 21614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:46.070851 21614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:46.077437 21614 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:39239
I20260808 10:11:46.077487 21747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:39239 every 8 connection(s)
I20260808 10:11:46.077899 21614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:11:46.083319 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.078657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:40139 (local address 127.2.30.190:46239)
0808 10:11:46.078734 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:46.078736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:46.079036 (+ 300us) server_negotiation.cc:407] Connection header received
0808 10:11:46.079572 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:46.079573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:46.079586 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:46.079643 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:46.080327 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.080831 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.081825 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.082012 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.082069 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.083039 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:46.083040 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:46.083040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:46.083046 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:46.083060 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:46.083064 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:46.083117 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:46.083225 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 10:11:46.083251 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:11:46.083583 21750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.078649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46239 (local address 127.2.30.129:40139)
0808 10:11:46.078892 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:46.078937 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:46.079476 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:46.079706 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:46.079709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:46.079718 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:46.080251 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:46.080255 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.080943 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.080944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:46.081740 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:46.081743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.082105 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.082106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:46.082219 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.082835 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:11:46.082835 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:46.082864 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:11:46.082903 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:46.083147 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:46.083153 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:46.083155 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:11:46.083213 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 10:11:46.083298 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":102,"threads_started":1}
I20260808 10:11:46.084175 21748 heartbeater.cc:347] Connected to a master server at 127.2.30.190:46239
I20260808 10:11:46.084285 21748 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:46.084424 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 21614
I20260808 10:11:46.084499 21748 heartbeater.cc:511] Master 127.2.30.190:46239 requested a full tablet report, sending...
I20260808 10:11:46.084540 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:11:46.084764 2170 external_mini_cluster.cc:1428] Setting key 55e84d0aa2888084120713d414bd29b0
I20260808 10:11:46.085160 21551 ts_manager.cc:194] Registered new tserver with Master: 8fe889eb21434fcf9862494251dc7324 (127.2.30.129:39239)
I20260808 10:11:46.085886 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46239
--builtin_ntp_servers=127.2.30.148:35855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:11:46.086084 21551 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.30.129:40139
W20260808 10:11:46.168982 21751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:46.169204 21751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:46.169227 21751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:46.170354 21751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:46.170396 21751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:46.170421 21751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:11:46.171756 21751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:46.171974 21751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:46.172240 21751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:46.177659 21757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:46.177659 21756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:46.177701 21759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:46.177815 21751 server_base.cc:1034] running on GCE node
I20260808 10:11:46.178138 21751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:46.178421 21751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:46.179581 21751 hybrid_clock.cc:648] HybridClock initialized: now 1786183906179559 us; error 40 us; skew 500 ppm
I20260808 10:11:46.180954 21751 webserver.cc:460] Webserver started at http://127.2.30.130:43929/ using document root <none> and password file <none>
I20260808 10:11:46.181217 21751 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:46.181327 21751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:46.181495 21751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:46.182613 21751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "f990d29ae02e4b1abc67195b248fa3a8"
format_stamp: "Formatted at 2026-08-08 10:11:46 on dist-test-slave-w206"
server_key: "64da685c38f41f41ae76502bfaa21d15"
server_key_iv: "882696299f218503ee7b1f4b632c8b52"
server_key_version: "encryptionkey@0"
I20260808 10:11:46.183048 21751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "f990d29ae02e4b1abc67195b248fa3a8"
format_stamp: "Formatted at 2026-08-08 10:11:46 on dist-test-slave-w206"
server_key: "64da685c38f41f41ae76502bfaa21d15"
server_key_iv: "882696299f218503ee7b1f4b632c8b52"
server_key_version: "encryptionkey@0"
I20260808 10:11:46.184484 21751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:46.185397 21765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:46.185597 21751 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:46.185729 21751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "f990d29ae02e4b1abc67195b248fa3a8"
format_stamp: "Formatted at 2026-08-08 10:11:46 on dist-test-slave-w206"
server_key: "64da685c38f41f41ae76502bfaa21d15"
server_key_iv: "882696299f218503ee7b1f4b632c8b52"
server_key_version: "encryptionkey@0"
I20260808 10:11:46.185823 21751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:46.199980 21751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:46.200357 21751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:46.200539 21751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:46.200830 21751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:46.201252 21751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:46.201321 21751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:46.201375 21751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:46.201416 21751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:46.208648 21751 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:35601
I20260808 10:11:46.208673 21885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:35601 every 8 connection(s)
I20260808 10:11:46.209111 21751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:11:46.211582 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 21751
I20260808 10:11:46.211717 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:11:46.211949 2170 external_mini_cluster.cc:1428] Setting key 4ef0427612de356b845c7a01d088373f
I20260808 10:11:46.213303 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46239
--builtin_ntp_servers=127.2.30.148:35855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:11:46.216218 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.210058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:42605 (local address 127.2.30.190:46239)
0808 10:11:46.210129 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:46.210131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:46.210420 (+ 289us) server_negotiation.cc:407] Connection header received
0808 10:11:46.211130 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:46.211132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:46.211149 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:46.211239 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:46.212314 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.213049 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.214560 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.214749 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.214815 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.215895 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:46.215897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:46.215898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:46.215904 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:46.215922 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:46.215927 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:46.215983 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:46.216082 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 10:11:46.216113 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:11:46.216629 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.210051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46239 (local address 127.2.30.130:42605)
0808 10:11:46.210275 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:46.210310 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:46.210802 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:46.211308 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:46.211311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:46.211328 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:46.212181 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:46.212188 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.213150 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.213151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:46.214464 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:46.214468 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.214847 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.214848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:46.215077 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.215744 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:11:46.215744 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:46.215774 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:11:46.215798 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:46.215999 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:46.216004 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:46.216007 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:11:46.216057 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 10:11:46.216193 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":112,"threads_started":1}
I20260808 10:11:46.217207 21886 heartbeater.cc:347] Connected to a master server at 127.2.30.190:46239
I20260808 10:11:46.217317 21886 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:46.217528 21886 heartbeater.cc:511] Master 127.2.30.190:46239 requested a full tablet report, sending...
I20260808 10:11:46.217916 21551 ts_manager.cc:194] Registered new tserver with Master: f990d29ae02e4b1abc67195b248fa3a8 (127.2.30.130:35601)
I20260808 10:11:46.218308 21551 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.30.130:42605
W20260808 10:11:46.304366 21889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:46.304631 21889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:46.304670 21889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:46.305889 21889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:46.305956 21889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:46.306001 21889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:11:46.307543 21889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:46239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:46.307832 21889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:46.308162 21889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:46.313714 21894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:46.313810 21895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:46.313740 21897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:46.313913 21889 server_base.cc:1034] running on GCE node
I20260808 10:11:46.314205 21889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:46.314504 21889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:46.315682 21889 hybrid_clock.cc:648] HybridClock initialized: now 1786183906315666 us; error 51 us; skew 500 ppm
I20260808 10:11:46.317138 21889 webserver.cc:460] Webserver started at http://127.2.30.131:36593/ using document root <none> and password file <none>
I20260808 10:11:46.317440 21889 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:46.317521 21889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:46.317678 21889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:46.318800 21889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "63128484905b4874bbe0fbbf0e3761ec"
format_stamp: "Formatted at 2026-08-08 10:11:46 on dist-test-slave-w206"
server_key: "0e85c9ce35b962b10582169f1c18ba9f"
server_key_iv: "1975bd9552effc81021333b5561efec2"
server_key_version: "encryptionkey@0"
I20260808 10:11:46.319186 21889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "63128484905b4874bbe0fbbf0e3761ec"
format_stamp: "Formatted at 2026-08-08 10:11:46 on dist-test-slave-w206"
server_key: "0e85c9ce35b962b10582169f1c18ba9f"
server_key_iv: "1975bd9552effc81021333b5561efec2"
server_key_version: "encryptionkey@0"
I20260808 10:11:46.320586 21889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:46.321359 21903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:46.321535 21889 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:11:46.321645 21889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "63128484905b4874bbe0fbbf0e3761ec"
format_stamp: "Formatted at 2026-08-08 10:11:46 on dist-test-slave-w206"
server_key: "0e85c9ce35b962b10582169f1c18ba9f"
server_key_iv: "1975bd9552effc81021333b5561efec2"
server_key_version: "encryptionkey@0"
I20260808 10:11:46.321722 21889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:46.333747 21889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:46.334101 21889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:46.334260 21889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:46.334499 21889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:46.334851 21889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:46.334908 21889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:46.334950 21889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:46.335004 21889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:46.341143 21889 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:35663
I20260808 10:11:46.341254 22022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:35663 every 8 connection(s)
I20260808 10:11:46.341867 21889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:11:46.347539 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.342696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:51637 (local address 127.2.30.190:46239)
0808 10:11:46.342780 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:46.342783 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:46.343093 (+ 310us) server_negotiation.cc:407] Connection header received
0808 10:11:46.343648 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:46.343649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:46.343660 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:46.343715 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:46.344393 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.344921 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.345980 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.346212 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.346273 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.347282 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:46.347283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:46.347283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:46.347288 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:46.347302 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:46.347306 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:46.347350 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:46.347442 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 10:11:46.347471 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:11:46.347733 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.342673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:46239 (local address 127.2.30.131:51637)
0808 10:11:46.342908 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:46.342967 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:46.343544 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:46.343771 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:46.343774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:46.343782 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:46.344307 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:46.344311 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.345030 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.345031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:46.345868 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:46.345871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.346310 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.346312 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:46.346434 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.347122 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:11:46.347123 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:46.347164 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:11:46.347195 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:46.347365 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:46.347371 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:46.347373 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:11:46.347427 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 10:11:46.347514 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":118,"threads_started":1}
I20260808 10:11:46.348246 22023 heartbeater.cc:347] Connected to a master server at 127.2.30.190:46239
I20260808 10:11:46.348338 22023 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:46.348547 22023 heartbeater.cc:511] Master 127.2.30.190:46239 requested a full tablet report, sending...
I20260808 10:11:46.348953 21551 ts_manager.cc:194] Registered new tserver with Master: 63128484905b4874bbe0fbbf0e3761ec (127.2.30.131:35663)
I20260808 10:11:46.349376 21551 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.30.131:51637
I20260808 10:11:46.350454 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 21889
I20260808 10:11:46.350546 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:11:46.350696 2170 external_mini_cluster.cc:1428] Setting key 24afe3e41f93489b2fa83cb5363290b5
I20260808 10:11:46.351577 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:11:46.355814 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.353019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47428 (local address 127.2.30.190:46239)
0808 10:11:46.353082 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:46.353085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:46.353180 (+ 95us) server_negotiation.cc:407] Connection header received
0808 10:11:46.353217 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:46.353217 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:46.353227 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:46.353275 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:46.353406 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.353875 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.354568 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.354742 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.354791 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.355571 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:46.355572 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:46.355572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:46.355576 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:46.355588 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:46.355591 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:46.355656 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:46.355749 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 10:11:46.355759 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 10:11:46.357540 21551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:11:46.359174 21551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:11:46.365990 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.361468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:35601 (local address 127.0.0.1:48806)
0808 10:11:46.361671 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:46.361715 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:46.361782 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:46.362054 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:46.362058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:46.362060 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:46.362149 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:46.362151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.363564 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.363565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:46.364854 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:46.364857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.365761 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.365762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:46.365827 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.365830 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:11:46.365883 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 10:11:46.365914 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":64,"thread_start_us":93,"threads_started":1}
I20260808 10:11:46.366017 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.361662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48806 (local address 127.2.30.130:35601)
0808 10:11:46.361874 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:46.361879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:46.361891 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:11:46.361932 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:46.361934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:46.361956 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:46.362020 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:46.362248 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.363491 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.364958 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.365676 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.365746 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.365829 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:46.365896 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 10:11:46.365933 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":81,"threads_started":1}
I20260808 10:11:46.366197 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.361650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49832 (local address 127.2.30.131:35663)
0808 10:11:46.362088 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:46.362092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:46.362101 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:11:46.362138 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:46.362140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:46.362155 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:46.362388 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:46.362453 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.363575 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.364775 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.365530 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.365614 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.365715 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:46.366044 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 10:11:46.366093 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":81,"threads_started":1}
I20260808 10:11:46.366000 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.361468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:35663 (local address 127.0.0.1:49832)
0808 10:11:46.361919 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:46.361929 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:46.361969 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:46.362270 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:46.362271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:46.362273 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:46.362316 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:46.362317 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.363638 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.363640 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:46.364675 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:46.364678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.365622 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.365623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:46.365684 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.365695 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 10:11:46.365814 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 10:11:46.365882 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":5,"thread_start_us":60,"threads_started":1}
I20260808 10:11:46.366866 21817 tablet_service.cc:1467] Processing CreateTablet for tablet 2172588462134209b439ee3a9a399179 (DEFAULT_TABLE table=test-table [id=614cda6ad4984e25b934bb1ccfc5371f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:46.367065 21953 tablet_service.cc:1467] Processing CreateTablet for tablet 2172588462134209b439ee3a9a399179 (DEFAULT_TABLE table=test-table [id=614cda6ad4984e25b934bb1ccfc5371f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:46.367318 21817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2172588462134209b439ee3a9a399179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:46.367443 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.361479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:39239 (local address 127.0.0.1:34242)
0808 10:11:46.362152 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:46.362164 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:46.362217 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:46.362490 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:46.362491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:46.362493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:46.362543 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:46.362544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.363499 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.363501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:46.364853 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:46.364857 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.367064 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.367066 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:46.367172 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.367177 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:11:46.367225 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 10:11:46.367328 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":54,"thread_start_us":120,"threads_started":1}
I20260808 10:11:46.367447 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.361748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34242 (local address 127.2.30.129:39239)
0808 10:11:46.361937 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:46.361944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:46.362281 (+ 337us) server_negotiation.cc:407] Connection header received
0808 10:11:46.362329 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:46.362331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:46.362347 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:46.362418 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:46.362623 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.363411 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.365773 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.366932 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.367061 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.367200 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:46.367268 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 10:11:46.367318 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":83,"threads_started":1}
I20260808 10:11:46.367455 21953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2172588462134209b439ee3a9a399179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:46.368137 21680 tablet_service.cc:1467] Processing CreateTablet for tablet 2172588462134209b439ee3a9a399179 (DEFAULT_TABLE table=test-table [id=614cda6ad4984e25b934bb1ccfc5371f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:46.368453 21680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2172588462134209b439ee3a9a399179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:46.370842 22038 tablet_bootstrap.cc:492] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8: Bootstrap starting.
I20260808 10:11:46.371121 22037 tablet_bootstrap.cc:492] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec: Bootstrap starting.
I20260808 10:11:46.371163 22039 tablet_bootstrap.cc:492] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324: Bootstrap starting.
I20260808 10:11:46.371759 22039 tablet_bootstrap.cc:654] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:46.371778 22037 tablet_bootstrap.cc:654] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:46.372066 22039 log.cc:824] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:46.372099 22037 log.cc:824] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:46.372807 22039 tablet_bootstrap.cc:492] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324: No bootstrap required, opened a new log
I20260808 10:11:46.372865 22037 tablet_bootstrap.cc:492] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec: No bootstrap required, opened a new log
I20260808 10:11:46.372895 22039 ts_tablet_manager.cc:1397] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 10:11:46.372936 22037 ts_tablet_manager.cc:1397] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 10:11:46.372983 22038 tablet_bootstrap.cc:654] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:46.373291 22038 log.cc:824] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:46.374184 22038 tablet_bootstrap.cc:492] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8: No bootstrap required, opened a new log
I20260808 10:11:46.374257 22038 ts_tablet_manager.cc:1397] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 10:11:46.375123 22039 raft_consensus.cc:348] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f990d29ae02e4b1abc67195b248fa3a8" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35601 } } peers { permanent_uuid: "63128484905b4874bbe0fbbf0e3761ec" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35663 } } peers { permanent_uuid: "8fe889eb21434fcf9862494251dc7324" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39239 } }
I20260808 10:11:46.375289 22039 raft_consensus.cc:374] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:46.375334 22039 raft_consensus.cc:729] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe889eb21434fcf9862494251dc7324, State: Initialized, Role: FOLLOWER
I20260808 10:11:46.375474 22039 consensus_queue.cc:260] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f990d29ae02e4b1abc67195b248fa3a8" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35601 } } peers { permanent_uuid: "63128484905b4874bbe0fbbf0e3761ec" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35663 } } peers { permanent_uuid: "8fe889eb21434fcf9862494251dc7324" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39239 } }
I20260808 10:11:46.375741 22039 ts_tablet_manager.cc:1428] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:11:46.375897 21748 heartbeater.cc:503] Master 127.2.30.190:46239 was elected leader, sending a full tablet report...
I20260808 10:11:46.376554 22037 raft_consensus.cc:348] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f990d29ae02e4b1abc67195b248fa3a8" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35601 } } peers { permanent_uuid: "63128484905b4874bbe0fbbf0e3761ec" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35663 } } peers { permanent_uuid: "8fe889eb21434fcf9862494251dc7324" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39239 } }
I20260808 10:11:46.376721 22037 raft_consensus.cc:374] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:46.376765 22037 raft_consensus.cc:729] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63128484905b4874bbe0fbbf0e3761ec, State: Initialized, Role: FOLLOWER
I20260808 10:11:46.376883 22037 consensus_queue.cc:260] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f990d29ae02e4b1abc67195b248fa3a8" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35601 } } peers { permanent_uuid: "63128484905b4874bbe0fbbf0e3761ec" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35663 } } peers { permanent_uuid: "8fe889eb21434fcf9862494251dc7324" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39239 } }
I20260808 10:11:46.376847 22038 raft_consensus.cc:348] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f990d29ae02e4b1abc67195b248fa3a8" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35601 } } peers { permanent_uuid: "63128484905b4874bbe0fbbf0e3761ec" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35663 } } peers { permanent_uuid: "8fe889eb21434fcf9862494251dc7324" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39239 } }
I20260808 10:11:46.376994 22038 raft_consensus.cc:374] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:46.377043 22038 raft_consensus.cc:729] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f990d29ae02e4b1abc67195b248fa3a8, State: Initialized, Role: FOLLOWER
I20260808 10:11:46.377146 22037 ts_tablet_manager.cc:1428] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 10:11:46.377161 22038 consensus_queue.cc:260] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f990d29ae02e4b1abc67195b248fa3a8" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35601 } } peers { permanent_uuid: "63128484905b4874bbe0fbbf0e3761ec" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35663 } } peers { permanent_uuid: "8fe889eb21434fcf9862494251dc7324" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39239 } }
I20260808 10:11:46.377215 22023 heartbeater.cc:503] Master 127.2.30.190:46239 was elected leader, sending a full tablet report...
I20260808 10:11:46.377460 22038 ts_tablet_manager.cc:1428] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:11:46.377503 21886 heartbeater.cc:503] Master 127.2.30.190:46239 was elected leader, sending a full tablet report...
I20260808 10:11:46.395862 22043 raft_consensus.cc:482] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:11:46.395998 22043 raft_consensus.cc:504] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f990d29ae02e4b1abc67195b248fa3a8" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35601 } } peers { permanent_uuid: "63128484905b4874bbe0fbbf0e3761ec" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35663 } } peers { permanent_uuid: "8fe889eb21434fcf9862494251dc7324" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39239 } }
I20260808 10:11:46.396310 22043 leader_election.cc:288] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f990d29ae02e4b1abc67195b248fa3a8 (127.2.30.130:35601), 63128484905b4874bbe0fbbf0e3761ec (127.2.30.131:35663)
I20260808 10:11:46.400660 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.396598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:49709 (local address 127.2.30.130:35601)
0808 10:11:46.396651 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:46.396654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:46.396665 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:11:46.396696 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:46.396697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:46.396707 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:46.396798 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:46.397013 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.397792 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.399667 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.400233 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.400318 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.400326 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:46.400543 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 10:11:46.400586 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 10:11:46.400660 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.396599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:48793 (local address 127.2.30.131:35663)
0808 10:11:46.396661 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:46.396663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:46.396778 (+ 115us) server_negotiation.cc:407] Connection header received
0808 10:11:46.396840 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:46.396841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:46.396850 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:46.396911 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:46.397068 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.397790 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.399665 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.400233 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.400318 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.400326 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:46.400543 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 10:11:46.400585 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 10:11:46.400679 21750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.396502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:35601 (local address 127.2.30.129:49709)
0808 10:11:46.396561 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:46.396574 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:46.396617 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:46.396827 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:46.396828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:46.396830 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:46.396930 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:46.396932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.397916 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.397918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:46.399548 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:46.399553 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.400363 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.400365 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:46.400451 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.400456 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:11:46.400523 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 10:11:46.400576 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":107}
I20260808 10:11:46.400686 22046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.396502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:35663 (local address 127.2.30.129:48793)
0808 10:11:46.396708 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:46.396716 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:46.396758 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:46.396925 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:46.396926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:46.396927 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:46.396995 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:46.396997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.397916 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.397918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:46.399552 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:46.399554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.400364 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.400365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:46.400456 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.400458 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 10:11:46.400523 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 10:11:46.400575 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":94,"thread_start_us":108,"threads_started":1}
I20260808 10:11:46.401094 21837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2172588462134209b439ee3a9a399179" candidate_uuid: "8fe889eb21434fcf9862494251dc7324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f990d29ae02e4b1abc67195b248fa3a8" is_pre_election: true
I20260808 10:11:46.401271 21837 raft_consensus.cc:2454] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe889eb21434fcf9862494251dc7324 in term 0.
I20260808 10:11:46.401206 21974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2172588462134209b439ee3a9a399179" candidate_uuid: "8fe889eb21434fcf9862494251dc7324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63128484905b4874bbe0fbbf0e3761ec" is_pre_election: true
I20260808 10:11:46.401387 21974 raft_consensus.cc:2454] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe889eb21434fcf9862494251dc7324 in term 0.
I20260808 10:11:46.401540 21634 leader_election.cc:302] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe889eb21434fcf9862494251dc7324, f990d29ae02e4b1abc67195b248fa3a8; no voters:
I20260808 10:11:46.401683 22043 raft_consensus.cc:2781] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:11:46.401742 22043 raft_consensus.cc:482] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:11:46.401779 22043 raft_consensus.cc:3037] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:46.402520 22043 raft_consensus.cc:504] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f990d29ae02e4b1abc67195b248fa3a8" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35601 } } peers { permanent_uuid: "63128484905b4874bbe0fbbf0e3761ec" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35663 } } peers { permanent_uuid: "8fe889eb21434fcf9862494251dc7324" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39239 } }
I20260808 10:11:46.402673 22043 leader_election.cc:288] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [CANDIDATE]: Term 1 election: Requested vote from peers f990d29ae02e4b1abc67195b248fa3a8 (127.2.30.130:35601), 63128484905b4874bbe0fbbf0e3761ec (127.2.30.131:35663)
I20260808 10:11:46.402808 21837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2172588462134209b439ee3a9a399179" candidate_uuid: "8fe889eb21434fcf9862494251dc7324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f990d29ae02e4b1abc67195b248fa3a8"
I20260808 10:11:46.402884 21837 raft_consensus.cc:3037] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:46.402906 21974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2172588462134209b439ee3a9a399179" candidate_uuid: "8fe889eb21434fcf9862494251dc7324" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63128484905b4874bbe0fbbf0e3761ec"
I20260808 10:11:46.403008 21974 raft_consensus.cc:3037] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:46.403748 21837 raft_consensus.cc:2454] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe889eb21434fcf9862494251dc7324 in term 1.
I20260808 10:11:46.403813 21974 raft_consensus.cc:2454] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe889eb21434fcf9862494251dc7324 in term 1.
I20260808 10:11:46.403959 21634 leader_election.cc:302] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fe889eb21434fcf9862494251dc7324, f990d29ae02e4b1abc67195b248fa3a8; no voters:
I20260808 10:11:46.404089 22043 raft_consensus.cc:2781] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:46.404256 22043 raft_consensus.cc:686] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe889eb21434fcf9862494251dc7324, State: Running, Role: LEADER
I20260808 10:11:46.404362 22043 consensus_queue.cc:237] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f990d29ae02e4b1abc67195b248fa3a8" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35601 } } peers { permanent_uuid: "63128484905b4874bbe0fbbf0e3761ec" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35663 } } peers { permanent_uuid: "8fe889eb21434fcf9862494251dc7324" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39239 } }
I20260808 10:11:46.405300 21551 catalog_manager.cc:5495] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe889eb21434fcf9862494251dc7324 (127.2.30.129). New cstate: current_term: 1 leader_uuid: "8fe889eb21434fcf9862494251dc7324" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f990d29ae02e4b1abc67195b248fa3a8" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63128484905b4874bbe0fbbf0e3761ec" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe889eb21434fcf9862494251dc7324" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 39239 } health_report { overall_health: HEALTHY } } }
I20260808 10:11:46.413371 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.410782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34256 (local address 127.2.30.129:39239)
0808 10:11:46.410834 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:46.410841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:46.410850 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:11:46.410953 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:46.410954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:46.411005 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:46.411073 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:11:46.411212 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.411681 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.412497 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:46.412699 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:46.412778 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:46.412937 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:11:46.413034 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:11:46.413147 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:46.413262 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 10:11:46.413275 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 10:11:46.415479 21837 raft_consensus.cc:1261] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8 [term 1 FOLLOWER]: Refusing update from remote peer 8fe889eb21434fcf9862494251dc7324: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:11:46.415819 21974 raft_consensus.cc:1261] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec [term 1 FOLLOWER]: Refusing update from remote peer 8fe889eb21434fcf9862494251dc7324: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:11:46.415920 22047 consensus_queue.cc:1035] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f990d29ae02e4b1abc67195b248fa3a8" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:11:46.416160 22047 consensus_queue.cc:1035] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63128484905b4874bbe0fbbf0e3761ec" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 35663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:11:46.419050 22050 mvcc.cc:205] Tried to move back new op lower bound from 7316209280674344960 to 7316209280633020416. Current Snapshot: MvccSnapshot[applied={T|T < 7316209280674344960}]
I20260808 10:11:46.419360 22051 mvcc.cc:205] Tried to move back new op lower bound from 7316209280674344960 to 7316209280633020416. Current Snapshot: MvccSnapshot[applied={T|T < 7316209280674344960 or (T in {7316209280674344960})}]
I20260808 10:11:46.421514 21551 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:11:46.421594 21551 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:11:46.422374 21551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d2de8c415c4e44e6a01d193fd8902fdf: Sending DeleteTablet for 3 replicas of tablet 2172588462134209b439ee3a9a399179
I20260808 10:11:46.422701 21817 tablet_service.cc:1514] Processing DeleteTablet for tablet 2172588462134209b439ee3a9a399179 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:46 UTC) from {username='slave'} at 127.0.0.1:48806
I20260808 10:11:46.422710 21953 tablet_service.cc:1514] Processing DeleteTablet for tablet 2172588462134209b439ee3a9a399179 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:46 UTC) from {username='slave'} at 127.0.0.1:49832
I20260808 10:11:46.422864 21680 tablet_service.cc:1514] Processing DeleteTablet for tablet 2172588462134209b439ee3a9a399179 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:46 UTC) from {username='slave'} at 127.0.0.1:34242
I20260808 10:11:46.423023 22061 tablet_replica.cc:330] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8: stopping tablet replica
I20260808 10:11:46.423120 22061 raft_consensus.cc:2229] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:11:46.423231 22061 raft_consensus.cc:2258] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:46.423281 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 21614
I20260808 10:11:46.423251 22062 tablet_replica.cc:330] T 2172588462134209b439ee3a9a399179 P 8fe889eb21434fcf9862494251dc7324: stopping tablet replica
I20260808 10:11:46.424841 22061 ts_tablet_manager.cc:1905] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:11:46.426419 22060 tablet_replica.cc:330] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec: stopping tablet replica
I20260808 10:11:46.426541 22060 raft_consensus.cc:2229] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:11:46.426649 22060 raft_consensus.cc:2258] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:46.427067 22061 ts_tablet_manager.cc:1918] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:11:46.427068 22060 ts_tablet_manager.cc:1905] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:11:46.427176 22061 log.cc:1196] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/2172588462134209b439ee3a9a399179
I20260808 10:11:46.427491 22061 ts_tablet_manager.cc:1939] T 2172588462134209b439ee3a9a399179 P f990d29ae02e4b1abc67195b248fa3a8: Deleting consensus metadata
I20260808 10:11:46.427970 21537 catalog_manager.cc:4841] TS f990d29ae02e4b1abc67195b248fa3a8 (127.2.30.130:35601): tablet 2172588462134209b439ee3a9a399179 (table test-table [id=614cda6ad4984e25b934bb1ccfc5371f]) successfully deleted
I20260808 10:11:46.428511 22060 ts_tablet_manager.cc:1918] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:11:46.428570 22060 log.cc:1196] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/2172588462134209b439ee3a9a399179
I20260808 10:11:46.428815 22060 ts_tablet_manager.cc:1939] T 2172588462134209b439ee3a9a399179 P 63128484905b4874bbe0fbbf0e3761ec: Deleting consensus metadata
I20260808 10:11:46.429292 21535 catalog_manager.cc:4841] TS 63128484905b4874bbe0fbbf0e3761ec (127.2.30.131:35663): tablet 2172588462134209b439ee3a9a399179 (table test-table [id=614cda6ad4984e25b934bb1ccfc5371f]) successfully deleted
W20260808 10:11:46.432291 21536 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:39239 (error 108)
I20260808 10:11:46.432514 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 21751
I20260808 10:11:46.432786 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:46.432667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:39239 (local address 127.0.0.1:34268)
0808 10:11:46.432715 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:46.432750 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:39239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 10:11:46.435747 21536 catalog_manager.cc:4584] TS 8fe889eb21434fcf9862494251dc7324 (127.2.30.129:39239): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2172588462134209b439ee3a9a399179: Network error: Client connection negotiation failed: client connection to 127.2.30.129:39239: connect: Connection refused (error 111)
I20260808 10:11:46.439632 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 21889
I20260808 10:11:46.445523 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 21517
2026-08-08T10:11:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (711 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:11:46 dist-test-slave-w206 krb5kdc[22066](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:11:46 dist-test-slave-w206 krb5kdc[22066](info): set up 2 sockets
Aug 08 10:11:46 dist-test-slave-w206 krb5kdc[22066](info): commencing operation
krb5kdc: starting...
W20260808 10:11:47.515304 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:11:47 dist-test-slave-w206 krb5kdc[22066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:11:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:11:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:11:47.650741 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:33845
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:33845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:47.735690 22080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:47.735966 22080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:47.736008 22080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:47.737181 22080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:11:47.737227 22080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:47.737241 22080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:47.737255 22080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:11:47.737267 22080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:11:47.738741 22080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:33845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:33845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:47.738953 22080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:47.739225 22080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:47.744621 22086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:47.744719 22088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:47.744886 22085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:47.744620 22080 server_base.cc:1034] running on GCE node
I20260808 10:11:47.745213 22080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:47.745527 22080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:47.746704 22080 hybrid_clock.cc:648] HybridClock initialized: now 1786183907746691 us; error 30 us; skew 500 ppm
Aug 08 10:11:47 dist-test-slave-w206 krb5kdc[22066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183907, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:47.749662 22080 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:11:47.750224 22080 webserver.cc:460] Webserver started at http://127.2.30.190:42107/ using document root <none> and password file <none>
I20260808 10:11:47.750464 22080 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:47.750535 22080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:47.750701 22080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:47.751910 22080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "c4870f9572e0403d8d1ffc49348780fc"
format_stamp: "Formatted at 2026-08-08 10:11:47 on dist-test-slave-w206"
server_key: "02952e74a776335ca79690b8d0029b4d"
server_key_iv: "158577994d92d75365d894b0c4a02c79"
server_key_version: "encryptionkey@0"
I20260808 10:11:47.752287 22080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "c4870f9572e0403d8d1ffc49348780fc"
format_stamp: "Formatted at 2026-08-08 10:11:47 on dist-test-slave-w206"
server_key: "02952e74a776335ca79690b8d0029b4d"
server_key_iv: "158577994d92d75365d894b0c4a02c79"
server_key_version: "encryptionkey@0"
I20260808 10:11:47.753698 22080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 10:11:47.754451 22095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:47.754635 22080 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:47.754762 22080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "c4870f9572e0403d8d1ffc49348780fc"
format_stamp: "Formatted at 2026-08-08 10:11:47 on dist-test-slave-w206"
server_key: "02952e74a776335ca79690b8d0029b4d"
server_key_iv: "158577994d92d75365d894b0c4a02c79"
server_key_version: "encryptionkey@0"
I20260808 10:11:47.754842 22080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:47.784868 22080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:47.787475 22080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:47.787652 22080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:47.792028 22080 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:33845
I20260808 10:11:47.792071 22149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:33845 every 8 connection(s)
I20260808 10:11:47.792451 22080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:11:47.793331 22150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:47.795866 22150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc: Bootstrap starting.
I20260808 10:11:47.796509 22150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:47.796836 22150 log.cc:824] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:47.797120 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 22080
I20260808 10:11:47.797313 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:11:47.797572 2170 external_mini_cluster.cc:1428] Setting key 28bf045e8d5c19768dbcba92fa28b167
I20260808 10:11:47.797753 22150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc: No bootstrap required, opened a new log
I20260808 10:11:47.800208 22150 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4870f9572e0403d8d1ffc49348780fc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 33845 } }
I20260808 10:11:47.800421 22150 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:47.800457 22150 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4870f9572e0403d8d1ffc49348780fc, State: Initialized, Role: FOLLOWER
I20260808 10:11:47.800604 22150 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4870f9572e0403d8d1ffc49348780fc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 33845 } }
I20260808 10:11:47.800699 22150 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:11:47.800733 22150 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:11:47.800770 22150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:47.801549 22150 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4870f9572e0403d8d1ffc49348780fc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 33845 } }
I20260808 10:11:47.801662 22150 leader_election.cc:302] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4870f9572e0403d8d1ffc49348780fc; no voters:
I20260808 10:11:47.801819 22150 leader_election.cc:288] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:11:47.801875 22155 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:47.802073 22155 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [term 1 LEADER]: Becoming Leader. State: Replica: c4870f9572e0403d8d1ffc49348780fc, State: Running, Role: LEADER
I20260808 10:11:47.802088 22150 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:11:47.802165 22155 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4870f9572e0403d8d1ffc49348780fc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 33845 } }
I20260808 10:11:47.802531 22155 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4870f9572e0403d8d1ffc49348780fc. Latest consensus state: current_term: 1 leader_uuid: "c4870f9572e0403d8d1ffc49348780fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4870f9572e0403d8d1ffc49348780fc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 33845 } } }
I20260808 10:11:47.802627 22155 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:47.802542 22156 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4870f9572e0403d8d1ffc49348780fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4870f9572e0403d8d1ffc49348780fc" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 33845 } } }
I20260808 10:11:47.802723 22156 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:47.803157 22162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:11:47.803690 22162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:11:47.805418 22162 catalog_manager.cc:1347] Generated new cluster ID: d8eb81d7f55348ff8c5703c58900cf66
I20260808 10:11:47.805470 22162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:11:47 dist-test-slave-w206 krb5kdc[22066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:11:47.810673 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:47.798160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45712 (local address 127.2.30.190:33845)
0808 10:11:47.798378 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:47.798396 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:47.798437 (+ 41us) server_negotiation.cc:407] Connection header received
0808 10:11:47.799320 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:47.799330 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:47.799396 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:47.799560 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:47.800219 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:47.802042 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:47.803323 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:47.804057 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:47.804167 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:47.806938 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:47.806942 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:47.806943 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:47.806955 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:47.808765 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:47.809163 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:47.809166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:47.809167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:47.809263 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:47.809494 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:47.809495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:47.809496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:47.809739 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:47.809918 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:47.810403 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 10:11:47.810518 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":100,"threads_started":1}
I20260808 10:11:47.815281 22162 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:11:47.815834 22162 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:11:47.827293 22162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc: Generated new TSK 0
I20260808 10:11:47.827549 22162 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.30.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
Entry for principal kudu/127.2.30.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab.
I20260808 10:11:47.864526 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:33845
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:47.988584 22177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:47.988894 22177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:47.988932 22177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:47.990459 22177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:47.990527 22177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:47.990566 22177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:11:47.992268 22177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.129.keytab
--principal=kudu/127.2.30.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:33845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:47.992477 22177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:47.992673 22177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:11:47.998054 22177 server_base.cc:1034] running on GCE node
W20260808 10:11:47.998026 22185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:47.998018 22187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:47.998018 22184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:47.998596 22177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:47.998852 22177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:48.000005 22177 hybrid_clock.cc:648] HybridClock initialized: now 1786183907999987 us; error 35 us; skew 500 ppm
Aug 08 10:11:47 dist-test-slave-w206 krb5kdc[22066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183907, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:48.002709 22177 init.cc:377] Logged in from keytab as kudu/127.2.30.129@KRBTEST.COM (short username kudu)
I20260808 10:11:48.003335 22177 webserver.cc:460] Webserver started at http://127.2.30.129:44325/ using document root <none> and password file <none>
I20260808 10:11:48.003603 22177 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:48.003674 22177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:48.003820 22177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:48.004902 22177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e"
format_stamp: "Formatted at 2026-08-08 10:11:47 on dist-test-slave-w206"
server_key: "712d499ff2727aebcd11c31519b9762e"
server_key_iv: "aa2801fbef49f797c4402517b8d87d88"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.005278 22177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e"
format_stamp: "Formatted at 2026-08-08 10:11:47 on dist-test-slave-w206"
server_key: "712d499ff2727aebcd11c31519b9762e"
server_key_iv: "aa2801fbef49f797c4402517b8d87d88"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.006671 22177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:11:48.007543 22194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:48.007741 22177 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:11:48.007838 22177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e"
format_stamp: "Formatted at 2026-08-08 10:11:47 on dist-test-slave-w206"
server_key: "712d499ff2727aebcd11c31519b9762e"
server_key_iv: "aa2801fbef49f797c4402517b8d87d88"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.007912 22177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:48.021781 22177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:48.024319 22177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:48.024499 22177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:48.024753 22177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:48.025117 22177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:48.025173 22177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:48.025210 22177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:48.025247 22177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:48.031411 22177 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:36555
I20260808 10:11:48.031490 22317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:36555 every 8 connection(s)
I20260808 10:11:48.031847 22177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:11:48.032095 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 22177
I20260808 10:11:48.032213 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:11:48.032508 2170 external_mini_cluster.cc:1428] Setting key 5b0763b5d85850c1e73be93f33935c04
Aug 08 10:11:48 dist-test-slave-w206 krb5kdc[22066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183907, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.129@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.30.130@KRBTEST.COM; defaulting to no policy
I20260808 10:11:48.043489 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.032715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:39675 (local address 127.2.30.190:33845)
0808 10:11:48.032765 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:48.032767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:48.033033 (+ 266us) server_negotiation.cc:407] Connection header received
0808 10:11:48.033802 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:48.033803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:48.033812 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:48.033864 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:48.035308 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.036135 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.037174 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.037412 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.037496 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.040194 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:48.040195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:48.040195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:48.040203 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:48.041788 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:48.042255 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:48.042256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:48.042257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:48.042324 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:48.042626 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:48.042628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:48.042628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:48.042762 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:48.042838 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:48.043234 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 10:11:48.043341 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:11:48.043619 22320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.032675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:33845 (local address 127.2.30.129:39675)
0808 10:11:48.032856 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:48.032930 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:48.033679 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:48.033944 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:48.033949 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:48.034838 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:48.035192 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.035200 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.036241 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.036242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.037070 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.037073 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.037509 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.037510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.037654 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.038268 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:11:48.038277 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:48.040078 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:48.041920 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:48.041928 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:48.041930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:48.042170 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:48.042404 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:48.042405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:48.042405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:48.042547 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:48.042841 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:48.042850 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:48.043081 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 10:11:48.043198 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 10:11:48.043372 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
I20260808 10:11:48.044126 22318 heartbeater.cc:347] Connected to a master server at 127.2.30.190:33845
I20260808 10:11:48.044226 22318 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:48.044380 22318 heartbeater.cc:511] Master 127.2.30.190:33845 requested a full tablet report, sending...
I20260808 10:11:48.044957 22114 ts_manager.cc:194] Registered new tserver with Master: 7e9b2f6dd21249e19d8a66d3b5ab677e (127.2.30.129:36555)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.130@KRBTEST.COM" created.
I20260808 10:11:48.045909 22114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.129@KRBTEST.COM'} at 127.2.30.129:39675
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
Entry for principal kudu/127.2.30.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab.
I20260808 10:11:48.062741 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:33845
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:48.148571 22324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:48.148823 22324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:48.148864 22324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:48.150081 22324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:48.150158 22324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:48.150206 22324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:11:48.151872 22324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.130.keytab
--principal=kudu/127.2.30.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:33845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:48.152154 22324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:48.152411 22324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:48.157922 22332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:48.157974 22329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:48.157955 22330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:48.158428 22324 server_base.cc:1034] running on GCE node
I20260808 10:11:48.158712 22324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:48.159055 22324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:48.160250 22324 hybrid_clock.cc:648] HybridClock initialized: now 1786183908160235 us; error 26 us; skew 500 ppm
Aug 08 10:11:48 dist-test-slave-w206 krb5kdc[22066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183908, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:48.163331 22324 init.cc:377] Logged in from keytab as kudu/127.2.30.130@KRBTEST.COM (short username kudu)
I20260808 10:11:48.163905 22324 webserver.cc:460] Webserver started at http://127.2.30.130:33613/ using document root <none> and password file <none>
I20260808 10:11:48.164157 22324 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:48.164230 22324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:48.164398 22324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:48.165572 22324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "512fc384e7e4469794cc8663fd43358d"
format_stamp: "Formatted at 2026-08-08 10:11:48 on dist-test-slave-w206"
server_key: "e7102b0174ee30be96103310b2dbeeab"
server_key_iv: "85f1d100373c2f10e4cb5b6c0316dad4"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.165961 22324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "512fc384e7e4469794cc8663fd43358d"
format_stamp: "Formatted at 2026-08-08 10:11:48 on dist-test-slave-w206"
server_key: "e7102b0174ee30be96103310b2dbeeab"
server_key_iv: "85f1d100373c2f10e4cb5b6c0316dad4"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.167531 22324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:11:48.168503 22339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:48.168704 22324 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:48.168802 22324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "512fc384e7e4469794cc8663fd43358d"
format_stamp: "Formatted at 2026-08-08 10:11:48 on dist-test-slave-w206"
server_key: "e7102b0174ee30be96103310b2dbeeab"
server_key_iv: "85f1d100373c2f10e4cb5b6c0316dad4"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.168866 22324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:48.182356 22324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:48.184800 22324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:48.184930 22324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:48.185153 22324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:48.185494 22324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:48.185529 22324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:48.185554 22324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:48.185597 22324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:48.191839 22324 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:36747
I20260808 10:11:48.191905 22460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:36747 every 8 connection(s)
I20260808 10:11:48.192279 22324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
Aug 08 10:11:48 dist-test-slave-w206 krb5kdc[22066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183908, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.130@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:11:48.199131 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 22324
I20260808 10:11:48.199280 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:11:48.199527 2170 external_mini_cluster.cc:1428] Setting key cd3a012b5ec41a94bc3a193a98f1c481
I20260808 10:11:48.202970 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.193132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:41605 (local address 127.2.30.190:33845)
0808 10:11:48.193215 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:48.193217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:48.193403 (+ 186us) server_negotiation.cc:407] Connection header received
0808 10:11:48.193897 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:48.193899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:48.193911 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:48.193961 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:48.194961 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.195490 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.196618 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.196804 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.196855 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.199439 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:48.199440 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:48.199440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:48.199446 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:48.201028 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:48.201642 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:48.201643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:48.201643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:48.201702 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:48.202040 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:48.202041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:48.202041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:48.202167 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:48.202244 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:48.202710 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 10:11:48.202850 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:11:48.203037 22463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.193081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:33845 (local address 127.2.30.130:41605)
0808 10:11:48.193273 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:48.193311 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:48.193811 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:48.194021 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:48.194024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:48.194670 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:48.194870 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.194875 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.195605 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.195606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.196529 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.196532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.196885 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.196886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.196990 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.197679 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:11:48.197688 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:48.199337 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:48.201120 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:48.201129 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:48.201132 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:48.201554 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:48.201771 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:48.201772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:48.201773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:48.201932 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:48.202260 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:48.202268 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:48.202506 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 10:11:48.202672 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 10:11:48.202813 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":100,"threads_started":1}
I20260808 10:11:48.203611 22461 heartbeater.cc:347] Connected to a master server at 127.2.30.190:33845
I20260808 10:11:48.203730 22461 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:48.203943 22461 heartbeater.cc:511] Master 127.2.30.190:33845 requested a full tablet report, sending...
I20260808 10:11:48.204375 22114 ts_manager.cc:194] Registered new tserver with Master: 512fc384e7e4469794cc8663fd43358d (127.2.30.130:36747)
I20260808 10:11:48.204773 22114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.130@KRBTEST.COM'} at 127.2.30.130:41605
WARNING: no policy specified for kudu/127.2.30.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
Entry for principal kudu/127.2.30.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab.
I20260808 10:11:48.229923 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:33845
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:48.319551 22466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:48.319808 22466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:48.319852 22466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:48.321105 22466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:48.321184 22466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:48.321234 22466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:11:48.322834 22466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.131.keytab
--principal=kudu/127.2.30.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:33845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:48.323168 22466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:48.323447 22466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:48.329192 22471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:48.329193 22474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:48.329339 22472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:48.329685 22466 server_base.cc:1034] running on GCE node
I20260808 10:11:48.329906 22466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:48.330171 22466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:48.331337 22466 hybrid_clock.cc:648] HybridClock initialized: now 1786183908331356 us; error 68 us; skew 500 ppm
Aug 08 10:11:48 dist-test-slave-w206 krb5kdc[22066](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183908, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:48.334174 22466 init.cc:377] Logged in from keytab as kudu/127.2.30.131@KRBTEST.COM (short username kudu)
I20260808 10:11:48.334777 22466 webserver.cc:460] Webserver started at http://127.2.30.131:43799/ using document root <none> and password file <none>
I20260808 10:11:48.335090 22466 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:48.335165 22466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:48.335314 22466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:48.336475 22466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "58ad7beec0bf4885aef56d5c48b55680"
format_stamp: "Formatted at 2026-08-08 10:11:48 on dist-test-slave-w206"
server_key: "1a1a8fd512d65a2ca29d77da103b0eda"
server_key_iv: "d05efa2c7634e6d343a9d5a6bd13ed63"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.336867 22466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "58ad7beec0bf4885aef56d5c48b55680"
format_stamp: "Formatted at 2026-08-08 10:11:48 on dist-test-slave-w206"
server_key: "1a1a8fd512d65a2ca29d77da103b0eda"
server_key_iv: "d05efa2c7634e6d343a9d5a6bd13ed63"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.338593 22466 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:11:48.339531 22481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:48.339747 22466 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:11:48.339864 22466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "58ad7beec0bf4885aef56d5c48b55680"
format_stamp: "Formatted at 2026-08-08 10:11:48 on dist-test-slave-w206"
server_key: "1a1a8fd512d65a2ca29d77da103b0eda"
server_key_iv: "d05efa2c7634e6d343a9d5a6bd13ed63"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.339962 22466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:48.355314 22466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:48.357580 22466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:48.357764 22466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:48.358035 22466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:48.358410 22466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:48.358486 22466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:48.358529 22466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:48.358563 22466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:48.365389 22466 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:37709
I20260808 10:11:48.365444 22599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:37709 every 8 connection(s)
I20260808 10:11:48.365823 22466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:11:48.369552 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 22466
I20260808 10:11:48.369674 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:11:48.369867 2170 external_mini_cluster.cc:1428] Setting key 3030a5ff38fc700688b75df03a1124f0
Aug 08 10:11:48 dist-test-slave-w206 krb5kdc[22066](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183908, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.131@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:11:48.377105 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.366782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:49531 (local address 127.2.30.190:33845)
0808 10:11:48.366844 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:48.366847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:48.367092 (+ 245us) server_negotiation.cc:407] Connection header received
0808 10:11:48.367632 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:48.367633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:48.367644 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:48.367701 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:48.368800 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.369320 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.371185 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.371395 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.371496 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.374363 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:48.374365 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:48.374365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:48.374411 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:48.375833 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:48.376263 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:48.376264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:48.376265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:48.376319 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:48.376547 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:48.376547 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:48.376547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:48.376658 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:48.376711 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:48.376968 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 10:11:48.377036 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 10:11:48.377363 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.366698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:33845 (local address 127.2.30.131:49531)
0808 10:11:48.366931 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:48.366984 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:48.367533 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:48.367801 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:48.367804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:48.368482 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:48.368690 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.368695 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.369455 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.369456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.371071 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.371080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.371488 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.371489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.371609 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.372359 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:11:48.372368 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:48.374266 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:48.375921 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:48.375930 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:48.375932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:48.376184 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:48.376380 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:11:48.376380 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:11:48.376381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:11:48.376492 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:11:48.376721 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:48.376726 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:48.376852 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 10:11:48.376942 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 10:11:48.377075 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":97,"threads_started":1}
I20260808 10:11:48.378024 22600 heartbeater.cc:347] Connected to a master server at 127.2.30.190:33845
I20260808 10:11:48.378163 22600 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:48.378474 22600 heartbeater.cc:511] Master 127.2.30.190:33845 requested a full tablet report, sending...
I20260808 10:11:48.379135 22114 ts_manager.cc:194] Registered new tserver with Master: 58ad7beec0bf4885aef56d5c48b55680 (127.2.30.131:37709)
I20260808 10:11:48.379593 22114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.30.131@KRBTEST.COM'} at 127.2.30.131:49531
I20260808 10:11:48.381946 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:11:48.389807 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.383530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45720 (local address 127.2.30.190:33845)
0808 10:11:48.383591 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:48.383593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:48.383666 (+ 73us) server_negotiation.cc:407] Connection header received
0808 10:11:48.383707 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:48.383708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:48.383717 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:48.383770 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:48.384358 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.384907 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.385672 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.385866 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.385931 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.387328 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:48.387329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:48.387329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:48.387337 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:48.388660 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:48.389042 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:48.389043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:48.389044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:48.389103 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:48.389303 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:48.389304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:48.389304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:48.389475 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:48.389563 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:48.389652 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 10:11:48.389726 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 10:11:48.391778 22114 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:11:48.393451 22114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:11:48.400728 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.396325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48996 (local address 127.2.30.131:37709)
0808 10:11:48.396526 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:48.396532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:48.396703 (+ 171us) server_negotiation.cc:407] Connection header received
0808 10:11:48.396773 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:48.396776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:48.396809 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:48.396889 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:48.397178 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.397862 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.399467 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.400248 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.400341 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.400414 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:48.400545 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 10:11:48.400595 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":84,"threads_started":1}
I20260808 10:11:48.400733 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.396015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:36555 (local address 127.0.0.1:55784)
0808 10:11:48.396215 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:48.396260 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:48.396331 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:48.396770 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:48.396773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:48.396776 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:48.396856 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.396857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.398253 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.398254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.399399 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.399401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.400423 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.400424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.400497 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.400502 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:11:48.400537 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 10:11:48.400622 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":4,"thread_start_us":85,"threads_started":1}
I20260808 10:11:48.400913 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.396242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55784 (local address 127.2.30.129:36555)
0808 10:11:48.396582 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:48.396586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:48.396596 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:11:48.396632 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:48.396634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:48.396648 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:48.396941 (+ 293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:48.397036 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.398153 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.399481 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.400325 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.400686 (+ 361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.400755 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:48.400783 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 10:11:48.400835 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":92,"threads_started":1}
I20260808 10:11:48.401036 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.396023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:37709 (local address 127.0.0.1:48996)
0808 10:11:48.396638 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:48.396645 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:48.396702 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:48.396997 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:48.396998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:48.397000 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:48.397056 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.397058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.397976 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.397978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.399350 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.399354 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.400339 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.400339 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.400428 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.400444 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 10:11:48.400520 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:11:48.400636 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":34,"thread_start_us":105,"threads_started":1}
I20260808 10:11:48.401753 22612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.396015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:36747 (local address 127.0.0.1:42310)
0808 10:11:48.396480 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:48.396492 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:48.396549 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:48.396844 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:48.396845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:48.396847 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:48.396887 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.396888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.399494 (+ 2606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.399494 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.400179 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.400180 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.401513 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.401515 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.401584 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.401588 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:11:48.401646 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 10:11:48.401689 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":4,"spinlock_wait_cycles":384,"thread_start_us":103,"threads_started":1}
I20260808 10:11:48.401626 22247 tablet_service.cc:1467] Processing CreateTablet for tablet 5b881630510f427b81483447e696cc0a (DEFAULT_TABLE table=test-table [id=76aac530b12a42f78db14345fb3bf404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:48.401793 22616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.396336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42310 (local address 127.2.30.130:36747)
0808 10:11:48.396536 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:48.396541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:48.396553 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:11:48.396623 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:48.396625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:48.396642 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:48.396761 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:48.396969 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.398137 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.400279 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.401338 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.401423 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.401476 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:48.401666 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 10:11:48.401714 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":108,"threads_started":1}
I20260808 10:11:48.401899 22532 tablet_service.cc:1467] Processing CreateTablet for tablet 5b881630510f427b81483447e696cc0a (DEFAULT_TABLE table=test-table [id=76aac530b12a42f78db14345fb3bf404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:48.402070 22247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b881630510f427b81483447e696cc0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:48.402258 22532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b881630510f427b81483447e696cc0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:48.402839 22392 tablet_service.cc:1467] Processing CreateTablet for tablet 5b881630510f427b81483447e696cc0a (DEFAULT_TABLE table=test-table [id=76aac530b12a42f78db14345fb3bf404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:48.403226 22392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b881630510f427b81483447e696cc0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:48.405287 22617 tablet_bootstrap.cc:492] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680: Bootstrap starting.
I20260808 10:11:48.405915 22618 tablet_bootstrap.cc:492] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e: Bootstrap starting.
I20260808 10:11:48.406015 22617 tablet_bootstrap.cc:654] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:48.406587 22618 tablet_bootstrap.cc:654] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:48.406620 22617 log.cc:824] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:48.406610 22619 tablet_bootstrap.cc:492] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d: Bootstrap starting.
I20260808 10:11:48.407186 22618 log.cc:824] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:48.407322 22619 tablet_bootstrap.cc:654] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:48.407557 22617 tablet_bootstrap.cc:492] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680: No bootstrap required, opened a new log
I20260808 10:11:48.407632 22617 ts_tablet_manager.cc:1397] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:11:48.407908 22619 log.cc:824] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:48.408041 22618 tablet_bootstrap.cc:492] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e: No bootstrap required, opened a new log
I20260808 10:11:48.408087 22618 ts_tablet_manager.cc:1397] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 10:11:48.408708 22619 tablet_bootstrap.cc:492] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d: No bootstrap required, opened a new log
I20260808 10:11:48.408764 22619 ts_tablet_manager.cc:1397] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 10:11:48.409556 22618 raft_consensus.cc:348] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512fc384e7e4469794cc8663fd43358d" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36747 } } peers { permanent_uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36555 } } peers { permanent_uuid: "58ad7beec0bf4885aef56d5c48b55680" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37709 } }
I20260808 10:11:48.409710 22618 raft_consensus.cc:374] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:48.409734 22618 raft_consensus.cc:729] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e9b2f6dd21249e19d8a66d3b5ab677e, State: Initialized, Role: FOLLOWER
I20260808 10:11:48.409731 22617 raft_consensus.cc:348] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512fc384e7e4469794cc8663fd43358d" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36747 } } peers { permanent_uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36555 } } peers { permanent_uuid: "58ad7beec0bf4885aef56d5c48b55680" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37709 } }
I20260808 10:11:48.409858 22618 consensus_queue.cc:260] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512fc384e7e4469794cc8663fd43358d" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36747 } } peers { permanent_uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36555 } } peers { permanent_uuid: "58ad7beec0bf4885aef56d5c48b55680" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37709 } }
I20260808 10:11:48.409898 22617 raft_consensus.cc:374] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:48.409934 22617 raft_consensus.cc:729] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58ad7beec0bf4885aef56d5c48b55680, State: Initialized, Role: FOLLOWER
I20260808 10:11:48.410039 22617 consensus_queue.cc:260] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512fc384e7e4469794cc8663fd43358d" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36747 } } peers { permanent_uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36555 } } peers { permanent_uuid: "58ad7beec0bf4885aef56d5c48b55680" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37709 } }
I20260808 10:11:48.410179 22618 ts_tablet_manager.cc:1428] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 10:11:48.410208 22318 heartbeater.cc:503] Master 127.2.30.190:33845 was elected leader, sending a full tablet report...
I20260808 10:11:48.410326 22617 ts_tablet_manager.cc:1428] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 10:11:48.410329 22600 heartbeater.cc:503] Master 127.2.30.190:33845 was elected leader, sending a full tablet report...
I20260808 10:11:48.411342 22619 raft_consensus.cc:348] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512fc384e7e4469794cc8663fd43358d" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36747 } } peers { permanent_uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36555 } } peers { permanent_uuid: "58ad7beec0bf4885aef56d5c48b55680" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37709 } }
I20260808 10:11:48.411557 22619 raft_consensus.cc:374] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:48.411610 22619 raft_consensus.cc:729] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512fc384e7e4469794cc8663fd43358d, State: Initialized, Role: FOLLOWER
I20260808 10:11:48.411729 22619 consensus_queue.cc:260] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512fc384e7e4469794cc8663fd43358d" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36747 } } peers { permanent_uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36555 } } peers { permanent_uuid: "58ad7beec0bf4885aef56d5c48b55680" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37709 } }
I20260808 10:11:48.412056 22619 ts_tablet_manager.cc:1428] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 10:11:48.412087 22461 heartbeater.cc:503] Master 127.2.30.190:33845 was elected leader, sending a full tablet report...
I20260808 10:11:48.421125 22623 raft_consensus.cc:482] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:11:48.421373 22623 raft_consensus.cc:504] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512fc384e7e4469794cc8663fd43358d" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36747 } } peers { permanent_uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36555 } } peers { permanent_uuid: "58ad7beec0bf4885aef56d5c48b55680" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37709 } }
I20260808 10:11:48.421813 22623 leader_election.cc:288] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 512fc384e7e4469794cc8663fd43358d (127.2.30.130:36747), 58ad7beec0bf4885aef56d5c48b55680 (127.2.30.131:37709)
I20260808 10:11:48.425634 22616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.422088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:33031 (local address 127.2.30.130:36747)
0808 10:11:48.422135 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:48.422137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:48.422172 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:11:48.422266 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:48.422268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:48.422280 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:48.422382 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:48.422688 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.423533 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.424827 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.425257 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.425317 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.425332 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:48.425503 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 10:11:48.425556 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:11:48.425684 22615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.422118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:38201 (local address 127.2.30.131:37709)
0808 10:11:48.422187 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:48.422189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:48.422334 (+ 145us) server_negotiation.cc:407] Connection header received
0808 10:11:48.422367 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:48.422368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:48.422378 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:48.422435 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:48.422688 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.423547 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.424869 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.425279 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.425379 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.425388 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:48.425563 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 10:11:48.425618 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 10:11:48.425684 22320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.422028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:36747 (local address 127.2.30.129:33031)
0808 10:11:48.422121 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:48.422134 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:48.422191 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:48.422444 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:48.422445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:48.422446 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:48.422571 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.422573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.423670 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.423672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.424727 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.424731 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.425343 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.425344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.425425 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.425428 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:11:48.425486 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 10:11:48.425551 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":24}
I20260808 10:11:48.425693 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.422042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:37709 (local address 127.2.30.129:38201)
0808 10:11:48.422249 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:48.422259 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:48.422326 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:48.422449 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:48.422450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:48.422452 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:48.422571 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.422573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.423703 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.423704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.424731 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.424734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.425393 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.425394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.425481 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.425486 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:11:48.425544 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 10:11:48.425579 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":23,"thread_start_us":135,"threads_started":1}
I20260808 10:11:48.426129 22554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b881630510f427b81483447e696cc0a" candidate_uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58ad7beec0bf4885aef56d5c48b55680" is_pre_election: true
I20260808 10:11:48.426230 22414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b881630510f427b81483447e696cc0a" candidate_uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "512fc384e7e4469794cc8663fd43358d" is_pre_election: true
I20260808 10:11:48.426373 22554 raft_consensus.cc:2454] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e9b2f6dd21249e19d8a66d3b5ab677e in term 0.
I20260808 10:11:48.426405 22414 raft_consensus.cc:2454] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e9b2f6dd21249e19d8a66d3b5ab677e in term 0.
I20260808 10:11:48.426623 22195 leader_election.cc:302] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58ad7beec0bf4885aef56d5c48b55680, 7e9b2f6dd21249e19d8a66d3b5ab677e; no voters:
I20260808 10:11:48.426783 22623 raft_consensus.cc:2781] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:11:48.426846 22623 raft_consensus.cc:482] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:11:48.426869 22623 raft_consensus.cc:3037] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:48.427652 22623 raft_consensus.cc:504] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512fc384e7e4469794cc8663fd43358d" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36747 } } peers { permanent_uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36555 } } peers { permanent_uuid: "58ad7beec0bf4885aef56d5c48b55680" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37709 } }
I20260808 10:11:48.427827 22623 leader_election.cc:288] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [CANDIDATE]: Term 1 election: Requested vote from peers 512fc384e7e4469794cc8663fd43358d (127.2.30.130:36747), 58ad7beec0bf4885aef56d5c48b55680 (127.2.30.131:37709)
I20260808 10:11:48.427997 22414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b881630510f427b81483447e696cc0a" candidate_uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "512fc384e7e4469794cc8663fd43358d"
I20260808 10:11:48.428077 22554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b881630510f427b81483447e696cc0a" candidate_uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58ad7beec0bf4885aef56d5c48b55680"
I20260808 10:11:48.428072 22414 raft_consensus.cc:3037] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:48.428174 22554 raft_consensus.cc:3037] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:48.428917 22414 raft_consensus.cc:2454] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e9b2f6dd21249e19d8a66d3b5ab677e in term 1.
I20260808 10:11:48.428937 22554 raft_consensus.cc:2454] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e9b2f6dd21249e19d8a66d3b5ab677e in term 1.
I20260808 10:11:48.429126 22197 leader_election.cc:302] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 512fc384e7e4469794cc8663fd43358d, 7e9b2f6dd21249e19d8a66d3b5ab677e; no voters:
I20260808 10:11:48.429242 22623 raft_consensus.cc:2781] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:48.429425 22623 raft_consensus.cc:686] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [term 1 LEADER]: Becoming Leader. State: Replica: 7e9b2f6dd21249e19d8a66d3b5ab677e, State: Running, Role: LEADER
I20260808 10:11:48.429533 22623 consensus_queue.cc:237] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512fc384e7e4469794cc8663fd43358d" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36747 } } peers { permanent_uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36555 } } peers { permanent_uuid: "58ad7beec0bf4885aef56d5c48b55680" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37709 } }
I20260808 10:11:48.430632 22114 catalog_manager.cc:5495] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e9b2f6dd21249e19d8a66d3b5ab677e (127.2.30.129). New cstate: current_term: 1 leader_uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "512fc384e7e4469794cc8663fd43358d" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e9b2f6dd21249e19d8a66d3b5ab677e" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 36555 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58ad7beec0bf4885aef56d5c48b55680" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37709 } health_report { overall_health: UNKNOWN } } }
I20260808 10:11:48.436271 22614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.433813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55786 (local address 127.2.30.129:36555)
0808 10:11:48.433879 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:48.433881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:48.433897 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:11:48.433929 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:48.433930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:48.433946 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:48.434012 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:11:48.434131 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.434638 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.435461 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.435677 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.435752 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.435925 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:11:48.436047 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:11:48.436145 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:48.436158 (+ 13us) server_negotiation.cc:299] Negotiation successful
0808 10:11:48.436181 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 10:11:48.438277 22554 raft_consensus.cc:1261] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680 [term 1 FOLLOWER]: Refusing update from remote peer 7e9b2f6dd21249e19d8a66d3b5ab677e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:11:48.438364 22414 raft_consensus.cc:1261] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d [term 1 FOLLOWER]: Refusing update from remote peer 7e9b2f6dd21249e19d8a66d3b5ab677e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:11:48.438710 22627 consensus_queue.cc:1035] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [LEADER]: Connected to new peer: Peer: permanent_uuid: "512fc384e7e4469794cc8663fd43358d" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 36747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:11:48.438894 22623 consensus_queue.cc:1035] T 5b881630510f427b81483447e696cc0a P 7e9b2f6dd21249e19d8a66d3b5ab677e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58ad7beec0bf4885aef56d5c48b55680" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 37709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:11:48.441960 22631 mvcc.cc:205] Tried to move back new op lower bound from 7316209288959967232 to 7316209288928075776. Current Snapshot: MvccSnapshot[applied={T|T < 7316209288959967232}]
I20260808 10:11:48.442443 22632 mvcc.cc:205] Tried to move back new op lower bound from 7316209288959967232 to 7316209288928075776. Current Snapshot: MvccSnapshot[applied={T|T < 7316209288959967232 or (T in {7316209288959967232})}]
I20260808 10:11:48.444615 22113 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:11:48.444736 22113 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:11:48.445639 22113 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4870f9572e0403d8d1ffc49348780fc: Sending DeleteTablet for 3 replicas of tablet 5b881630510f427b81483447e696cc0a
I20260808 10:11:48.446060 22247 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b881630510f427b81483447e696cc0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:48 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:55784
I20260808 10:11:48.446066 22532 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b881630510f427b81483447e696cc0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:48 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:48996
I20260808 10:11:48.446226 22392 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b881630510f427b81483447e696cc0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:48 UTC) from {username='kudu', principal='kudu/127.2.30.190@KRBTEST.COM'} at 127.0.0.1:42310
I20260808 10:11:48.446602 22641 tablet_replica.cc:330] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d: stopping tablet replica
I20260808 10:11:48.446653 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 22177
I20260808 10:11:48.446724 22641 raft_consensus.cc:2229] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:11:48.447953 22641 raft_consensus.cc:2258] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:48.448324 22640 tablet_replica.cc:330] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680: stopping tablet replica
I20260808 10:11:48.448458 22640 raft_consensus.cc:2229] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:11:48.448578 22640 raft_consensus.cc:2258] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:48.448927 22641 ts_tablet_manager.cc:1905] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:11:48.449146 22640 ts_tablet_manager.cc:1905] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:11:48.451061 22640 ts_tablet_manager.cc:1918] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:11:48.451119 22641 ts_tablet_manager.cc:1918] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:11:48.451149 22640 log.cc:1196] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/5b881630510f427b81483447e696cc0a
I20260808 10:11:48.451206 22641 log.cc:1196] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/5b881630510f427b81483447e696cc0a
I20260808 10:11:48.451458 22640 ts_tablet_manager.cc:1939] T 5b881630510f427b81483447e696cc0a P 58ad7beec0bf4885aef56d5c48b55680: Deleting consensus metadata
I20260808 10:11:48.451511 22641 ts_tablet_manager.cc:1939] T 5b881630510f427b81483447e696cc0a P 512fc384e7e4469794cc8663fd43358d: Deleting consensus metadata
I20260808 10:11:48.452081 22096 catalog_manager.cc:4841] TS 58ad7beec0bf4885aef56d5c48b55680 (127.2.30.131:37709): tablet 5b881630510f427b81483447e696cc0a (table test-table [id=76aac530b12a42f78db14345fb3bf404]) successfully deleted
I20260808 10:11:48.452081 22097 catalog_manager.cc:4841] TS 512fc384e7e4469794cc8663fd43358d (127.2.30.130:36747): tablet 5b881630510f427b81483447e696cc0a (table test-table [id=76aac530b12a42f78db14345fb3bf404]) successfully deleted
W20260808 10:11:48.454807 22100 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:36555 (error 108)
I20260808 10:11:48.455449 22612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.455298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:36555 (local address 127.0.0.1:55802)
0808 10:11:48.455364 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:48.455408 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:36555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 10:11:48.455453 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 22324
W20260808 10:11:48.457134 22100 catalog_manager.cc:4584] TS 7e9b2f6dd21249e19d8a66d3b5ab677e (127.2.30.129:36555): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b881630510f427b81483447e696cc0a: Network error: Client connection negotiation failed: client connection to 127.2.30.129:36555: connect: Connection refused (error 111)
I20260808 10:11:48.465224 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 22466
I20260808 10:11:48.471414 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 22080
2026-08-08T10:11:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2028 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T10:11:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:11:48Z Disabled control of system clock
I20260808 10:11:48.504510 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38857
--webserver_interface=127.2.30.190
--webserver_port=0
--builtin_ntp_servers=127.2.30.148:34115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:38857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:11:48.592028 22646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:48.592283 22646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:48.592305 22646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:48.593469 22646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:11:48.593510 22646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:48.593523 22646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:48.593535 22646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:11:48.593549 22646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:11:48.594941 22646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:34115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:38857
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:38857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:48.595191 22646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:48.595458 22646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:48.601029 22656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:48.601054 22654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:48.601135 22653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:48.601580 22646 server_base.cc:1034] running on GCE node
I20260808 10:11:48.601801 22646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:48.602169 22646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:48.603332 22646 hybrid_clock.cc:648] HybridClock initialized: now 1786183908603317 us; error 31 us; skew 500 ppm
I20260808 10:11:48.604750 22646 webserver.cc:460] Webserver started at http://127.2.30.190:39241/ using document root <none> and password file <none>
I20260808 10:11:48.605017 22646 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:48.605067 22646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:48.605221 22646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:48.606406 22646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "3ca5604ca9cf44c48f3c45c9aabd4ba2"
format_stamp: "Formatted at 2026-08-08 10:11:48 on dist-test-slave-w206"
server_key: "0b36cf4d2a5d5134118b1334a4f6383f"
server_key_iv: "c186edf8a73144263b50e2f833211395"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.606799 22646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "3ca5604ca9cf44c48f3c45c9aabd4ba2"
format_stamp: "Formatted at 2026-08-08 10:11:48 on dist-test-slave-w206"
server_key: "0b36cf4d2a5d5134118b1334a4f6383f"
server_key_iv: "c186edf8a73144263b50e2f833211395"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.608263 22646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:48.609090 22663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:48.609419 22646 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:48.609546 22646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "3ca5604ca9cf44c48f3c45c9aabd4ba2"
format_stamp: "Formatted at 2026-08-08 10:11:48 on dist-test-slave-w206"
server_key: "0b36cf4d2a5d5134118b1334a4f6383f"
server_key_iv: "c186edf8a73144263b50e2f833211395"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.609628 22646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:48.633309 22646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:48.633701 22646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:48.633877 22646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:48.638381 22646 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:38857
I20260808 10:11:48.638414 22717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:38857 every 8 connection(s)
I20260808 10:11:48.638809 22646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:11:48.639916 22718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:48.640429 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 22646
I20260808 10:11:48.640612 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:11:48.640815 2170 external_mini_cluster.cc:1428] Setting key 211ce56700777b1e3ba1391e8edc1215
I20260808 10:11:48.643187 22718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2: Bootstrap starting.
I20260808 10:11:48.643895 22718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:48.644233 22718 log.cc:824] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:48.645057 22718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2: No bootstrap required, opened a new log
I20260808 10:11:48.647184 22721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.641486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47916 (local address 127.2.30.190:38857)
0808 10:11:48.641720 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:48.641728 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:48.641759 (+ 31us) server_negotiation.cc:407] Connection header received
0808 10:11:48.642187 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:48.642193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:48.642254 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:48.642378 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:48.642675 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.643718 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.644831 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.645294 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.645407 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.646145 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:48.646149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:48.646149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:48.646393 (+ 244us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:48.646449 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:48.646469 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:48.646564 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:48.646666 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 10:11:48.646736 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":116,"threads_started":1}
I20260808 10:11:48.647241 22718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca5604ca9cf44c48f3c45c9aabd4ba2" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38857 } }
I20260808 10:11:48.647452 22718 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:48.647503 22718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ca5604ca9cf44c48f3c45c9aabd4ba2, State: Initialized, Role: FOLLOWER
I20260808 10:11:48.647639 22718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca5604ca9cf44c48f3c45c9aabd4ba2" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38857 } }
I20260808 10:11:48.647720 22718 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:11:48.647758 22718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:11:48.647806 22718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:48.648545 22718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca5604ca9cf44c48f3c45c9aabd4ba2" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38857 } }
I20260808 10:11:48.648670 22718 leader_election.cc:302] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ca5604ca9cf44c48f3c45c9aabd4ba2; no voters:
I20260808 10:11:48.648876 22718 leader_election.cc:288] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:11:48.648921 22723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:48.649047 22723 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [term 1 LEADER]: Becoming Leader. State: Replica: 3ca5604ca9cf44c48f3c45c9aabd4ba2, State: Running, Role: LEADER
I20260808 10:11:48.649135 22723 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca5604ca9cf44c48f3c45c9aabd4ba2" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38857 } }
I20260808 10:11:48.649185 22718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:11:48.649492 22724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ca5604ca9cf44c48f3c45c9aabd4ba2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca5604ca9cf44c48f3c45c9aabd4ba2" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38857 } } }
I20260808 10:11:48.649492 22725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ca5604ca9cf44c48f3c45c9aabd4ba2. Latest consensus state: current_term: 1 leader_uuid: "3ca5604ca9cf44c48f3c45c9aabd4ba2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ca5604ca9cf44c48f3c45c9aabd4ba2" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 38857 } } }
I20260808 10:11:48.649710 22725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:48.649966 22724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2 [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:48.650025 22730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:11:48.650807 22730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:11:48.652558 22730 catalog_manager.cc:1347] Generated new cluster ID: b58ecc5ebc1c4525a572438c818afb12
I20260808 10:11:48.652616 22730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:11:48.669073 22730 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:11:48.669888 22730 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:11:48.674357 22730 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2: Generated new TSK 0
I20260808 10:11:48.674606 22730 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:11:48.682621 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.129:0
--local_ip_for_outbound_sockets=127.2.30.129
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38857
--builtin_ntp_servers=127.2.30.148:34115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 10:11:48.772085 22743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:48.772329 22743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:48.772379 22743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:48.773591 22743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:48.773661 22743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:48.773706 22743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.129
I20260808 10:11:48.775233 22743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:34115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.30.129
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.129
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:48.775520 22743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:48.775840 22743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:48.781307 22749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:48.781312 22751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:48.781329 22748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:48.781481 22743 server_base.cc:1034] running on GCE node
I20260808 10:11:48.781769 22743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:48.782032 22743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:48.783203 22743 hybrid_clock.cc:648] HybridClock initialized: now 1786183908783194 us; error 42 us; skew 500 ppm
I20260808 10:11:48.784566 22743 webserver.cc:460] Webserver started at http://127.2.30.129:42851/ using document root <none> and password file <none>
I20260808 10:11:48.784842 22743 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:48.784945 22743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:48.785108 22743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:48.786222 22743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/data/instance:
uuid: "793d1647e7164f3389ae8f30cc98432f"
format_stamp: "Formatted at 2026-08-08 10:11:48 on dist-test-slave-w206"
server_key: "4e1f1e8e00293ebc054e4b70a816cae0"
server_key_iv: "16a3c6c78ffdf16abce038f779e44c48"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.786609 22743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance:
uuid: "793d1647e7164f3389ae8f30cc98432f"
format_stamp: "Formatted at 2026-08-08 10:11:48 on dist-test-slave-w206"
server_key: "4e1f1e8e00293ebc054e4b70a816cae0"
server_key_iv: "16a3c6c78ffdf16abce038f779e44c48"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.788121 22743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:11:48.789016 22757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:48.789201 22743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:48.789310 22743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/wal
uuid: "793d1647e7164f3389ae8f30cc98432f"
format_stamp: "Formatted at 2026-08-08 10:11:48 on dist-test-slave-w206"
server_key: "4e1f1e8e00293ebc054e4b70a816cae0"
server_key_iv: "16a3c6c78ffdf16abce038f779e44c48"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.789388 22743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:48.797040 22743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:48.797377 22743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:48.797528 22743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:48.797767 22743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:48.798125 22743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:48.798184 22743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:48.798249 22743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:48.798292 22743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:48.804862 22743 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.129:45643
I20260808 10:11:48.804914 22875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.129:45643 every 8 connection(s)
I20260808 10:11:48.805308 22743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/data/info.pb
I20260808 10:11:48.808873 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 22743
I20260808 10:11:48.808987 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-0/wal/instance
I20260808 10:11:48.809206 2170 external_mini_cluster.cc:1428] Setting key 643534a42a0314962f64615a823ce0ca
I20260808 10:11:48.810340 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.130:0
--local_ip_for_outbound_sockets=127.2.30.130
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38857
--builtin_ntp_servers=127.2.30.148:34115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:11:48.811473 22721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.806065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.129:33169 (local address 127.2.30.190:38857)
0808 10:11:48.806118 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:48.806124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:48.806372 (+ 248us) server_negotiation.cc:407] Connection header received
0808 10:11:48.806929 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:48.806930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:48.806947 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:48.807043 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:48.807771 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.808301 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.809775 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.810048 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.810102 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.811162 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:48.811163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:48.811164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:48.811168 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:48.811184 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:48.811188 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:48.811292 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:48.811348 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 10:11:48.811384 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:11:48.811830 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.806048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38857 (local address 127.2.30.129:33169)
0808 10:11:48.806241 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:48.806276 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:48.806817 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:48.807118 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:48.807122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:48.807132 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:48.807661 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.807665 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.808457 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.808458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.809691 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.809694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.810129 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.810130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.810305 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.810955 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:11:48.810956 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:48.811020 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:11:48.811052 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:48.811262 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:48.811269 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:48.811272 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:11:48.811347 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 10:11:48.811477 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260808 10:11:48.812449 22876 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38857
I20260808 10:11:48.812626 22876 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:48.812881 22876 heartbeater.cc:511] Master 127.2.30.190:38857 requested a full tablet report, sending...
I20260808 10:11:48.813431 22680 ts_manager.cc:194] Registered new tserver with Master: 793d1647e7164f3389ae8f30cc98432f (127.2.30.129:45643)
I20260808 10:11:48.814079 22680 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.30.129:33169
W20260808 10:11:48.895767 22879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:48.896013 22879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:48.896050 22879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:48.897257 22879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:48.897329 22879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:48.897382 22879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.130
I20260808 10:11:48.898787 22879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:34115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.30.130
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.130
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:48.899080 22879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:48.899412 22879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:48.904690 22887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:48.904665 22885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:48.904726 22884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:48.904947 22879 server_base.cc:1034] running on GCE node
I20260808 10:11:48.905150 22879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:48.905344 22879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:48.906495 22879 hybrid_clock.cc:648] HybridClock initialized: now 1786183908906479 us; error 30 us; skew 500 ppm
I20260808 10:11:48.907842 22879 webserver.cc:460] Webserver started at http://127.2.30.130:33297/ using document root <none> and password file <none>
I20260808 10:11:48.908097 22879 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:48.908166 22879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:48.908322 22879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:48.909464 22879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/data/instance:
uuid: "6859d19b9ff94f8f97eb75bcff88b011"
format_stamp: "Formatted at 2026-08-08 10:11:48 on dist-test-slave-w206"
server_key: "48e182a76f969dc6fea6a9d4ff0932c3"
server_key_iv: "3deb209d197fcfcbc7b96f668b0a84b6"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.909835 22879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance:
uuid: "6859d19b9ff94f8f97eb75bcff88b011"
format_stamp: "Formatted at 2026-08-08 10:11:48 on dist-test-slave-w206"
server_key: "48e182a76f969dc6fea6a9d4ff0932c3"
server_key_iv: "3deb209d197fcfcbc7b96f668b0a84b6"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.911351 22879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:11:48.912201 22893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:48.912426 22879 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:48.912545 22879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/wal
uuid: "6859d19b9ff94f8f97eb75bcff88b011"
format_stamp: "Formatted at 2026-08-08 10:11:48 on dist-test-slave-w206"
server_key: "48e182a76f969dc6fea6a9d4ff0932c3"
server_key_iv: "3deb209d197fcfcbc7b96f668b0a84b6"
server_key_version: "encryptionkey@0"
I20260808 10:11:48.912629 22879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:48.924032 22879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:48.924355 22879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:48.924472 22879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:48.924696 22879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:48.925016 22879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:48.925056 22879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:48.925083 22879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:48.925107 22879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:48.931725 22879 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.130:35829
I20260808 10:11:48.931818 23011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.130:35829 every 8 connection(s)
I20260808 10:11:48.932231 22879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/data/info.pb
I20260808 10:11:48.937028 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 22879
I20260808 10:11:48.937144 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/wal/instance
I20260808 10:11:48.937439 2170 external_mini_cluster.cc:1428] Setting key 62cba88d45bcb7ecd48c83fed52318e9
I20260808 10:11:48.938186 22721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.933122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:32819 (local address 127.2.30.190:38857)
0808 10:11:48.933291 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:48.933295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:48.933477 (+ 182us) server_negotiation.cc:407] Connection header received
0808 10:11:48.933974 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:48.933975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:48.933987 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:48.934053 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:48.934789 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.935442 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.936470 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.936665 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.936737 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.937855 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:48.937857 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:48.937857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:48.937862 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:48.937886 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:48.937891 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:48.937952 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:48.938087 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 10:11:48.938116 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260808 10:11:48.938454 23014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:48.933082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38857 (local address 127.2.30.130:32819)
0808 10:11:48.933312 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:48.933358 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:48.933883 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:48.934116 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:48.934119 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:48.934128 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:48.934678 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.934682 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.935565 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.935566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.936367 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:48.936370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:48.936761 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:48.936762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:48.936902 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:48.937660 (+ 758us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:11:48.937661 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:48.937693 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:11:48.937717 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:48.937968 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:48.937975 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:48.937978 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:11:48.938053 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 10:11:48.938180 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":110,"threads_started":1}
I20260808 10:11:48.938726 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.30.131:0
--local_ip_for_outbound_sockets=127.2.30.131
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38857
--builtin_ntp_servers=127.2.30.148:34115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 10:11:48.939145 23012 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38857
I20260808 10:11:48.939286 23012 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:48.939507 23012 heartbeater.cc:511] Master 127.2.30.190:38857 requested a full tablet report, sending...
I20260808 10:11:48.939970 22680 ts_manager.cc:194] Registered new tserver with Master: 6859d19b9ff94f8f97eb75bcff88b011 (127.2.30.130:35829)
I20260808 10:11:48.940513 22680 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.30.130:32819
W20260808 10:11:49.030903 23015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:49.031239 23015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:49.031279 23015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:49.032553 23015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:49.032639 23015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:49.032683 23015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.30.131
I20260808 10:11:49.034178 23015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:34115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.30.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.30.131
--webserver_port=0
--tserver_master_addrs=127.2.30.190:38857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.30.131
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:49.034513 23015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:49.034869 23015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:49.041059 23021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:49.041077 23020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:49.041267 23015 server_base.cc:1034] running on GCE node
W20260808 10:11:49.041081 23023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:49.041622 23015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:49.041904 23015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:49.043121 23015 hybrid_clock.cc:648] HybridClock initialized: now 1786183909043091 us; error 38 us; skew 500 ppm
I20260808 10:11:49.044653 23015 webserver.cc:460] Webserver started at http://127.2.30.131:43687/ using document root <none> and password file <none>
I20260808 10:11:49.044988 23015 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:49.045069 23015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:49.045235 23015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:49.046396 23015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/data/instance:
uuid: "f704a44c558f4d928c5b4ea90fed37ef"
format_stamp: "Formatted at 2026-08-08 10:11:49 on dist-test-slave-w206"
server_key: "ddf19c33e66252c001daadb1ac4e03f7"
server_key_iv: "820f5fe7819f54561ad4ba77105e2b8a"
server_key_version: "encryptionkey@0"
I20260808 10:11:49.046792 23015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance:
uuid: "f704a44c558f4d928c5b4ea90fed37ef"
format_stamp: "Formatted at 2026-08-08 10:11:49 on dist-test-slave-w206"
server_key: "ddf19c33e66252c001daadb1ac4e03f7"
server_key_iv: "820f5fe7819f54561ad4ba77105e2b8a"
server_key_version: "encryptionkey@0"
I20260808 10:11:49.048683 23015 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260808 10:11:49.049751 23029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:49.049978 23015 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:49.050107 23015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/wal
uuid: "f704a44c558f4d928c5b4ea90fed37ef"
format_stamp: "Formatted at 2026-08-08 10:11:49 on dist-test-slave-w206"
server_key: "ddf19c33e66252c001daadb1ac4e03f7"
server_key_iv: "820f5fe7819f54561ad4ba77105e2b8a"
server_key_version: "encryptionkey@0"
I20260808 10:11:49.050210 23015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:49.064170 23015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:49.064586 23015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:49.064774 23015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:49.065066 23015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 10:11:49.065464 23015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 10:11:49.065528 23015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:49.065568 23015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 10:11:49.065623 23015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:49.072758 23015 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.131:38437
I20260808 10:11:49.072814 23146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.131:38437 every 8 connection(s)
I20260808 10:11:49.073227 23015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/data/info.pb
I20260808 10:11:49.076087 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 23015
I20260808 10:11:49.076231 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/wal/instance
I20260808 10:11:49.076501 2170 external_mini_cluster.cc:1428] Setting key f7dbb619cc4878ea2bf0879b866429dd
I20260808 10:11:49.080304 22721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:49.074179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.131:42393 (local address 127.2.30.190:38857)
0808 10:11:49.074326 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:49.074329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:49.074572 (+ 243us) server_negotiation.cc:407] Connection header received
0808 10:11:49.075246 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:49.075248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:49.075261 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:49.075321 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:49.076170 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.076867 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.078456 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.078716 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.078807 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:49.079932 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:49.079933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:49.079934 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:49.079940 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:49.079957 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:49.079962 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:49.080032 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:49.080155 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 10:11:49.080192 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260808 10:11:49.080631 23150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:49.074084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:38857 (local address 127.2.30.131:42393)
0808 10:11:49.074425 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:49.074469 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:49.075133 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:49.075477 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:49.075481 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:49.075489 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:11:49.076060 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:49.076065 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.076979 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.076980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:49.078320 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:49.078324 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.078822 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.078824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:49.079118 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:49.079717 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 10:11:49.079718 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:11:49.079753 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 10:11:49.079781 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:11:49.080047 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:11:49.080055 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:11:49.080059 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 10:11:49.080157 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 10:11:49.080287 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":120,"threads_started":1}
I20260808 10:11:49.081398 23147 heartbeater.cc:347] Connected to a master server at 127.2.30.190:38857
I20260808 10:11:49.081502 23147 heartbeater.cc:464] Registering TS with master...
I20260808 10:11:49.081676 23147 heartbeater.cc:511] Master 127.2.30.190:38857 requested a full tablet report, sending...
I20260808 10:11:49.082170 22680 ts_manager.cc:194] Registered new tserver with Master: f704a44c558f4d928c5b4ea90fed37ef (127.2.30.131:38437)
I20260808 10:11:49.082698 22680 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.30.131:42393
I20260808 10:11:49.089414 2170 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 10:11:49.094482 22721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:49.091086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47932 (local address 127.2.30.190:38857)
0808 10:11:49.091165 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:49.091167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:49.091404 (+ 237us) server_negotiation.cc:407] Connection header received
0808 10:11:49.091457 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:49.091457 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:49.091469 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:49.091523 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:49.091745 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.092319 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.093223 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.093438 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.093506 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:49.094163 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:49.094164 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:49.094165 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:49.094170 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:49.094185 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:49.094189 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:49.094242 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:49.094284 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 10:11:49.094308 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:11:49.096639 22680 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 10:11:49.098567 22680 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 10:11:49.106702 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:49.102141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37210 (local address 127.2.30.130:35829)
0808 10:11:49.102338 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:49.102345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:49.102357 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:11:49.102406 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:49.102408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:49.102635 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:49.102937 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:49.103049 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.103972 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.105376 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.106193 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.106270 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:49.106374 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:49.106549 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 10:11:49.106600 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":100,"threads_started":1}
I20260808 10:11:49.106699 23159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:49.101757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.130:35829 (local address 127.0.0.1:37210)
0808 10:11:49.101892 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:49.101939 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:49.102010 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:49.102759 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:49.102762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:49.102765 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:49.102850 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:49.102853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.104058 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.104060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:49.105260 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:49.105263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.106294 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.106295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:49.106417 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:49.106435 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 10:11:49.106512 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 10:11:49.106579 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260808 10:11:49.107044 23160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:49.102141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:45643 (local address 127.0.0.1:44306)
0808 10:11:49.102342 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:49.102353 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:49.102602 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:49.103321 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:49.103322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:49.103324 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:49.103369 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:49.103370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.104703 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.104704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:49.105638 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:49.105641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.106553 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.106554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:49.106609 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:49.106612 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 10:11:49.106660 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 10:11:49.106694 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":116,"threads_started":1}
I20260808 10:11:49.107400 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:49.102791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44306 (local address 127.2.30.129:45643)
0808 10:11:49.103101 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:49.103106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:49.103115 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:11:49.103154 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:49.103156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:49.103171 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:49.103248 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:49.103439 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.104575 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.105746 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.106347 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.106418 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:49.106516 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:49.106747 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 10:11:49.107020 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":117,"threads_started":1}
I20260808 10:11:49.107599 23162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:49.102004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:38437 (local address 127.0.0.1:50916)
0808 10:11:49.102321 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:49.102331 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:49.102384 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:49.103237 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:49.103237 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:49.103240 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:49.103290 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:49.103292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.104613 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.104614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:49.105687 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:49.105689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.106662 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.106663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:49.106740 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:49.106745 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:11:49.106778 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 10:11:49.106830 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":93,"threads_started":1}
I20260808 10:11:49.107913 22941 tablet_service.cc:1467] Processing CreateTablet for tablet 7d34e7dc829a401fbb06cb0daef130f8 (DEFAULT_TABLE table=test-table [id=dfad812b673d4fbcae630735d99b8f41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:49.108086 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:49.102372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50916 (local address 127.2.30.131:38437)
0808 10:11:49.103065 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:49.103069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:49.103080 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:11:49.103122 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:49.103123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:49.103137 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:49.103237 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:49.103376 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.104509 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.105786 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.106560 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.107808 (+ 1248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:49.107927 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:49.107956 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 10:11:49.108005 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":270,"threads_started":1}
I20260808 10:11:49.108157 22807 tablet_service.cc:1467] Processing CreateTablet for tablet 7d34e7dc829a401fbb06cb0daef130f8 (DEFAULT_TABLE table=test-table [id=dfad812b673d4fbcae630735d99b8f41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:49.108378 22941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d34e7dc829a401fbb06cb0daef130f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:49.108530 22807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d34e7dc829a401fbb06cb0daef130f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:49.108712 23077 tablet_service.cc:1467] Processing CreateTablet for tablet 7d34e7dc829a401fbb06cb0daef130f8 (DEFAULT_TABLE table=test-table [id=dfad812b673d4fbcae630735d99b8f41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 10:11:49.109074 23077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d34e7dc829a401fbb06cb0daef130f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:49.113504 23165 tablet_bootstrap.cc:492] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011: Bootstrap starting.
I20260808 10:11:49.113873 23166 tablet_bootstrap.cc:492] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef: Bootstrap starting.
I20260808 10:11:49.114583 23165 tablet_bootstrap.cc:654] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:49.114666 23167 tablet_bootstrap.cc:492] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f: Bootstrap starting.
I20260808 10:11:49.114962 23165 log.cc:824] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:49.114960 23166 tablet_bootstrap.cc:654] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:49.115525 23167 tablet_bootstrap.cc:654] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:49.115664 23166 log.cc:824] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:49.115965 23167 log.cc:824] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:49.116953 23165 tablet_bootstrap.cc:492] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011: No bootstrap required, opened a new log
I20260808 10:11:49.116953 23167 tablet_bootstrap.cc:492] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f: No bootstrap required, opened a new log
I20260808 10:11:49.117040 23165 ts_tablet_manager.cc:1397] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260808 10:11:49.117040 23167 ts_tablet_manager.cc:1397] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 10:11:49.117061 23166 tablet_bootstrap.cc:492] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef: No bootstrap required, opened a new log
I20260808 10:11:49.117158 23166 ts_tablet_manager.cc:1397] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 10:11:49.120074 23166 raft_consensus.cc:348] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859d19b9ff94f8f97eb75bcff88b011" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35829 } } peers { permanent_uuid: "793d1647e7164f3389ae8f30cc98432f" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45643 } } peers { permanent_uuid: "f704a44c558f4d928c5b4ea90fed37ef" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 38437 } }
I20260808 10:11:49.120071 23165 raft_consensus.cc:348] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859d19b9ff94f8f97eb75bcff88b011" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35829 } } peers { permanent_uuid: "793d1647e7164f3389ae8f30cc98432f" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45643 } } peers { permanent_uuid: "f704a44c558f4d928c5b4ea90fed37ef" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 38437 } }
I20260808 10:11:49.120071 23167 raft_consensus.cc:348] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859d19b9ff94f8f97eb75bcff88b011" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35829 } } peers { permanent_uuid: "793d1647e7164f3389ae8f30cc98432f" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45643 } } peers { permanent_uuid: "f704a44c558f4d928c5b4ea90fed37ef" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 38437 } }
I20260808 10:11:49.120314 23165 raft_consensus.cc:374] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:49.120314 23167 raft_consensus.cc:374] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:49.120316 23166 raft_consensus.cc:374] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:49.120357 23167 raft_consensus.cc:729] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 793d1647e7164f3389ae8f30cc98432f, State: Initialized, Role: FOLLOWER
I20260808 10:11:49.120357 23165 raft_consensus.cc:729] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6859d19b9ff94f8f97eb75bcff88b011, State: Initialized, Role: FOLLOWER
I20260808 10:11:49.120358 23166 raft_consensus.cc:729] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f704a44c558f4d928c5b4ea90fed37ef, State: Initialized, Role: FOLLOWER
I20260808 10:11:49.120504 23165 consensus_queue.cc:260] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859d19b9ff94f8f97eb75bcff88b011" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35829 } } peers { permanent_uuid: "793d1647e7164f3389ae8f30cc98432f" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45643 } } peers { permanent_uuid: "f704a44c558f4d928c5b4ea90fed37ef" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 38437 } }
I20260808 10:11:49.120514 23167 consensus_queue.cc:260] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859d19b9ff94f8f97eb75bcff88b011" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35829 } } peers { permanent_uuid: "793d1647e7164f3389ae8f30cc98432f" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45643 } } peers { permanent_uuid: "f704a44c558f4d928c5b4ea90fed37ef" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 38437 } }
I20260808 10:11:49.120576 23166 consensus_queue.cc:260] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859d19b9ff94f8f97eb75bcff88b011" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35829 } } peers { permanent_uuid: "793d1647e7164f3389ae8f30cc98432f" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45643 } } peers { permanent_uuid: "f704a44c558f4d928c5b4ea90fed37ef" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 38437 } }
I20260808 10:11:49.120842 23167 ts_tablet_manager.cc:1428] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 10:11:49.120867 23165 ts_tablet_manager.cc:1428] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260808 10:11:49.120910 23166 ts_tablet_manager.cc:1428] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 10:11:49.120990 22876 heartbeater.cc:503] Master 127.2.30.190:38857 was elected leader, sending a full tablet report...
I20260808 10:11:49.121018 23012 heartbeater.cc:503] Master 127.2.30.190:38857 was elected leader, sending a full tablet report...
I20260808 10:11:49.121294 23147 heartbeater.cc:503] Master 127.2.30.190:38857 was elected leader, sending a full tablet report...
W20260808 10:11:49.182940 23013 tablet.cc:2369] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:11:49.306097 22877 tablet.cc:2369] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 10:11:49.323942 23148 tablet.cc:2369] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:11:49.357420 23172 raft_consensus.cc:482] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 10:11:49.357574 23172 raft_consensus.cc:504] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859d19b9ff94f8f97eb75bcff88b011" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35829 } } peers { permanent_uuid: "793d1647e7164f3389ae8f30cc98432f" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45643 } } peers { permanent_uuid: "f704a44c558f4d928c5b4ea90fed37ef" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 38437 } }
I20260808 10:11:49.357967 23172 leader_election.cc:288] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 793d1647e7164f3389ae8f30cc98432f (127.2.30.129:45643), f704a44c558f4d928c5b4ea90fed37ef (127.2.30.131:38437)
I20260808 10:11:49.363132 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:49.358485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:56801 (local address 127.2.30.131:38437)
0808 10:11:49.358562 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:49.358565 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:49.358577 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:11:49.358613 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:49.358614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:49.358625 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:49.358711 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:49.359210 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.360113 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.361985 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.362590 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.362689 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:49.362700 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:49.362995 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 10:11:49.363052 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 10:11:49.363132 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:49.358513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.30.130:55383 (local address 127.2.30.129:45643)
0808 10:11:49.358592 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:49.358595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:49.358686 (+ 91us) server_negotiation.cc:407] Connection header received
0808 10:11:49.358760 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:49.358761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:49.358772 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:49.358835 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 10:11:49.359222 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.360113 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.361985 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.362590 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.362695 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:49.362702 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:49.362996 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 10:11:49.363044 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 10:11:49.363147 23014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:49.358248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.131:38437 (local address 127.2.30.130:56801)
0808 10:11:49.358338 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:49.358351 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:49.358401 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:49.358747 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:49.358748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:49.358750 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:49.358881 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:49.358884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.360368 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.360370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:49.361808 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:49.361814 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.362744 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.362746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:49.362850 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:49.362855 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:11:49.362934 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 10:11:49.363040 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":115}
I20260808 10:11:49.363152 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:49.358248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:45643 (local address 127.2.30.130:55383)
0808 10:11:49.358572 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:49.358586 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:11:49.358674 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:11:49.358858 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:11:49.358859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:11:49.358860 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 10:11:49.358920 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:49.358922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.360369 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.360370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:49.361808 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:11:49.361814 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.362752 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.362753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:11:49.362850 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:49.362855 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 10:11:49.362934 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 10:11:49.363032 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":84,"thread_start_us":116,"threads_started":1}
I20260808 10:11:49.363703 23098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d34e7dc829a401fbb06cb0daef130f8" candidate_uuid: "6859d19b9ff94f8f97eb75bcff88b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f704a44c558f4d928c5b4ea90fed37ef" is_pre_election: true
I20260808 10:11:49.363927 23098 raft_consensus.cc:2454] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6859d19b9ff94f8f97eb75bcff88b011 in term 0.
I20260808 10:11:49.363900 22827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d34e7dc829a401fbb06cb0daef130f8" candidate_uuid: "6859d19b9ff94f8f97eb75bcff88b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "793d1647e7164f3389ae8f30cc98432f" is_pre_election: true
I20260808 10:11:49.364032 22827 raft_consensus.cc:2454] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6859d19b9ff94f8f97eb75bcff88b011 in term 0.
I20260808 10:11:49.364238 22897 leader_election.cc:302] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6859d19b9ff94f8f97eb75bcff88b011, f704a44c558f4d928c5b4ea90fed37ef; no voters:
I20260808 10:11:49.364480 23172 raft_consensus.cc:2781] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 10:11:49.364550 23172 raft_consensus.cc:482] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 10:11:49.364578 23172 raft_consensus.cc:3037] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:49.365430 23172 raft_consensus.cc:504] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859d19b9ff94f8f97eb75bcff88b011" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35829 } } peers { permanent_uuid: "793d1647e7164f3389ae8f30cc98432f" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45643 } } peers { permanent_uuid: "f704a44c558f4d928c5b4ea90fed37ef" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 38437 } }
I20260808 10:11:49.365604 23172 leader_election.cc:288] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [CANDIDATE]: Term 1 election: Requested vote from peers 793d1647e7164f3389ae8f30cc98432f (127.2.30.129:45643), f704a44c558f4d928c5b4ea90fed37ef (127.2.30.131:38437)
I20260808 10:11:49.365833 23098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d34e7dc829a401fbb06cb0daef130f8" candidate_uuid: "6859d19b9ff94f8f97eb75bcff88b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f704a44c558f4d928c5b4ea90fed37ef"
I20260808 10:11:49.365866 22827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d34e7dc829a401fbb06cb0daef130f8" candidate_uuid: "6859d19b9ff94f8f97eb75bcff88b011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "793d1647e7164f3389ae8f30cc98432f"
I20260808 10:11:49.365942 23098 raft_consensus.cc:3037] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:49.365967 22827 raft_consensus.cc:3037] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:49.366786 22827 raft_consensus.cc:2454] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6859d19b9ff94f8f97eb75bcff88b011 in term 1.
I20260808 10:11:49.367056 22895 leader_election.cc:302] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6859d19b9ff94f8f97eb75bcff88b011, 793d1647e7164f3389ae8f30cc98432f; no voters:
I20260808 10:11:49.367170 23172 raft_consensus.cc:2781] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:49.367245 23098 raft_consensus.cc:2454] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6859d19b9ff94f8f97eb75bcff88b011 in term 1.
I20260808 10:11:49.367358 23172 raft_consensus.cc:686] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [term 1 LEADER]: Becoming Leader. State: Replica: 6859d19b9ff94f8f97eb75bcff88b011, State: Running, Role: LEADER
I20260808 10:11:49.367463 23172 consensus_queue.cc:237] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859d19b9ff94f8f97eb75bcff88b011" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35829 } } peers { permanent_uuid: "793d1647e7164f3389ae8f30cc98432f" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45643 } } peers { permanent_uuid: "f704a44c558f4d928c5b4ea90fed37ef" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 38437 } }
I20260808 10:11:49.368386 22679 catalog_manager.cc:5495] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6859d19b9ff94f8f97eb75bcff88b011 (127.2.30.130). New cstate: current_term: 1 leader_uuid: "6859d19b9ff94f8f97eb75bcff88b011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6859d19b9ff94f8f97eb75bcff88b011" member_type: VOTER last_known_addr { host: "127.2.30.130" port: 35829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "793d1647e7164f3389ae8f30cc98432f" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f704a44c558f4d928c5b4ea90fed37ef" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 38437 } health_report { overall_health: UNKNOWN } } }
I20260808 10:11:49.394546 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:49.391519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37216 (local address 127.2.30.130:35829)
0808 10:11:49.391589 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:49.391591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:49.391599 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:11:49.391641 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:49.391642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:49.391665 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:49.391731 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:11:49.391872 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.392457 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.393507 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:49.393717 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:49.393784 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:49.393976 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:11:49.394089 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:11:49.394158 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:49.394370 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 10:11:49.394398 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 10:11:49.396819 23098 raft_consensus.cc:1261] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef [term 1 FOLLOWER]: Refusing update from remote peer 6859d19b9ff94f8f97eb75bcff88b011: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:11:49.396835 22827 raft_consensus.cc:1261] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f [term 1 FOLLOWER]: Refusing update from remote peer 6859d19b9ff94f8f97eb75bcff88b011: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:11:49.397311 23175 consensus_queue.cc:1035] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "793d1647e7164f3389ae8f30cc98432f" member_type: VOTER last_known_addr { host: "127.2.30.129" port: 45643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:11:49.397490 23172 consensus_queue.cc:1035] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f704a44c558f4d928c5b4ea90fed37ef" member_type: VOTER last_known_addr { host: "127.2.30.131" port: 38437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:11:49.400473 23178 mvcc.cc:205] Tried to move back new op lower bound from 7316209292885999616 to 7316209292769746944. Current Snapshot: MvccSnapshot[applied={T|T < 7316209292885999616}]
I20260808 10:11:49.400902 23180 mvcc.cc:205] Tried to move back new op lower bound from 7316209292885999616 to 7316209292769746944. Current Snapshot: MvccSnapshot[applied={T|T < 7316209292885999616}]
I20260808 10:11:49.401366 23179 mvcc.cc:205] Tried to move back new op lower bound from 7316209292885999616 to 7316209292769746944. Current Snapshot: MvccSnapshot[applied={T|T < 7316209292885999616 or (T in {7316209292885999616})}]
I20260808 10:11:49.404110 22679 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:11:49.404244 22679 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 10:11:49.405170 22679 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ca5604ca9cf44c48f3c45c9aabd4ba2: Sending DeleteTablet for 3 replicas of tablet 7d34e7dc829a401fbb06cb0daef130f8
I20260808 10:11:49.405602 22941 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d34e7dc829a401fbb06cb0daef130f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:49 UTC) from {username='slave'} at 127.0.0.1:37210
I20260808 10:11:49.405599 23077 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d34e7dc829a401fbb06cb0daef130f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:49 UTC) from {username='slave'} at 127.0.0.1:50916
I20260808 10:11:49.405933 22807 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d34e7dc829a401fbb06cb0daef130f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 10:11:49 UTC) from {username='slave'} at 127.0.0.1:44306
I20260808 10:11:49.406210 23191 tablet_replica.cc:330] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f: stopping tablet replica
I20260808 10:11:49.406222 23189 tablet_replica.cc:330] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011: stopping tablet replica
I20260808 10:11:49.406315 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 22743
I20260808 10:11:49.406347 23189 raft_consensus.cc:2229] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [term 1 LEADER]: Raft consensus shutting down.
I20260808 10:11:49.406463 23191 raft_consensus.cc:2229] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:11:49.406504 23189 raft_consensus.cc:2258] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:49.406705 23191 raft_consensus.cc:2258] T 7d34e7dc829a401fbb06cb0daef130f8 P 793d1647e7164f3389ae8f30cc98432f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:49.407162 23189 ts_tablet_manager.cc:1905] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 10:11:49.409062 23189 ts_tablet_manager.cc:1918] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:11:49.409209 23189 log.cc:1196] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-1/wal/wals/7d34e7dc829a401fbb06cb0daef130f8
I20260808 10:11:49.409519 23189 ts_tablet_manager.cc:1939] T 7d34e7dc829a401fbb06cb0daef130f8 P 6859d19b9ff94f8f97eb75bcff88b011: Deleting consensus metadata
I20260808 10:11:49.410060 22667 catalog_manager.cc:4841] TS 6859d19b9ff94f8f97eb75bcff88b011 (127.2.30.130:35829): tablet 7d34e7dc829a401fbb06cb0daef130f8 (table test-table [id=dfad812b673d4fbcae630735d99b8f41]) successfully deleted
I20260808 10:11:49.415336 23188 tablet_replica.cc:330] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef: stopping tablet replica
I20260808 10:11:49.415581 23188 raft_consensus.cc:2229] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 10:11:49.415736 23188 raft_consensus.cc:2258] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 10:11:49.416313 23188 ts_tablet_manager.cc:1905] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 10:11:49.417179 22665 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.30.129:45643 (error 108)
I20260808 10:11:49.417357 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 22879
I20260808 10:11:49.421916 23162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:49.421682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.129:45643 (local address 127.0.0.1:44320)
0808 10:11:49.421777 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 10:11:49.421855 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.129:45643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 10:11:49.422113 22665 catalog_manager.cc:4584] TS 793d1647e7164f3389ae8f30cc98432f (127.2.30.129:45643): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7d34e7dc829a401fbb06cb0daef130f8: Network error: Client connection negotiation failed: client connection to 127.2.30.129:45643: connect: Connection refused (error 111)
I20260808 10:11:49.423709 23188 ts_tablet_manager.cc:1918] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 10:11:49.423830 23188 log.cc:1196] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef: Deleting WAL directory at /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786183651681294-2170-0/minicluster-data/ts-2/wal/wals/7d34e7dc829a401fbb06cb0daef130f8
I20260808 10:11:49.424124 23188 ts_tablet_manager.cc:1939] T 7d34e7dc829a401fbb06cb0daef130f8 P f704a44c558f4d928c5b4ea90fed37ef: Deleting consensus metadata
I20260808 10:11:49.425014 22667 catalog_manager.cc:4841] TS f704a44c558f4d928c5b4ea90fed37ef (127.2.30.131:38437): tablet 7d34e7dc829a401fbb06cb0daef130f8 (table test-table [id=dfad812b673d4fbcae630735d99b8f41]) successfully deleted
I20260808 10:11:49.427894 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 23015
I20260808 10:11:49.434864 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 22646
2026-08-08T10:11:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (963 ms)
[----------] 6 tests from EncryptionPolicyTest (8692 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:11:49 dist-test-slave-w206 krb5kdc[23196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:11:49 dist-test-slave-w206 krb5kdc[23196](info): set up 2 sockets
Aug 08 10:11:49 dist-test-slave-w206 krb5kdc[23196](info): commencing operation
krb5kdc: starting...
W20260808 10:11:50.547067 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:11:50 dist-test-slave-w206 krb5kdc[23196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:11:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:11:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:11:50.682690 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44603
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:50.768774 23209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:50.769016 23209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:50.769042 23209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:50.770200 23209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:11:50.770246 23209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:50.770262 23209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:50.770274 23209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:11:50.770287 23209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:11:50.771860 23209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44603
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:50.772082 23209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:50.772281 23209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:50.777575 23215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:50.777772 23209 server_base.cc:1034] running on GCE node
W20260808 10:11:50.777601 23217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:50.777583 23214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:50.778224 23209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:50.778635 23209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:50.779805 23209 hybrid_clock.cc:648] HybridClock initialized: now 1786183910779790 us; error 48 us; skew 500 ppm
Aug 08 10:11:50 dist-test-slave-w206 krb5kdc[23196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183910, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:50.782467 23209 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:11:50.783044 23209 webserver.cc:460] Webserver started at http://127.0.0.1:39853/ using document root <none> and password file <none>
I20260808 10:11:50.783289 23209 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:50.783366 23209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:50.783529 23209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:50.784622 23209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "2a357d91efc046d0952170248b0c868f"
format_stamp: "Formatted at 2026-08-08 10:11:50 on dist-test-slave-w206"
server_key: "4ae9932139edf53f42e6bf4622f82c2e"
server_key_iv: "153fab868461374bdd1ab97451ce2bb2"
server_key_version: "encryptionkey@0"
I20260808 10:11:50.784997 23209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "2a357d91efc046d0952170248b0c868f"
format_stamp: "Formatted at 2026-08-08 10:11:50 on dist-test-slave-w206"
server_key: "4ae9932139edf53f42e6bf4622f82c2e"
server_key_iv: "153fab868461374bdd1ab97451ce2bb2"
server_key_version: "encryptionkey@0"
I20260808 10:11:50.786466 23209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:11:50.787289 23224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:50.787539 23209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:50.787652 23209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "2a357d91efc046d0952170248b0c868f"
format_stamp: "Formatted at 2026-08-08 10:11:50 on dist-test-slave-w206"
server_key: "4ae9932139edf53f42e6bf4622f82c2e"
server_key_iv: "153fab868461374bdd1ab97451ce2bb2"
server_key_version: "encryptionkey@0"
I20260808 10:11:50.787734 23209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:50.800237 23209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:50.802454 23209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:50.802656 23209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:50.807298 23209 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44603
I20260808 10:11:50.807361 23278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44603 every 8 connection(s)
I20260808 10:11:50.807763 23209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:11:50.808976 23279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:50.808996 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 23209
I20260808 10:11:50.809113 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:11:50.809365 2170 external_mini_cluster.cc:1428] Setting key 60c3b90b13c7df1568cc956c08d20604
I20260808 10:11:50.812342 23279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f: Bootstrap starting.
I20260808 10:11:50.813033 23279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:50.813333 23279 log.cc:824] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:50.814075 23279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f: No bootstrap required, opened a new log
Aug 08 10:11:50 dist-test-slave-w206 krb5kdc[23196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:11:50.816246 23279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a357d91efc046d0952170248b0c868f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44603 } }
I20260808 10:11:50.816442 23279 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:50.816493 23279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a357d91efc046d0952170248b0c868f, State: Initialized, Role: FOLLOWER
I20260808 10:11:50.816668 23279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a357d91efc046d0952170248b0c868f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44603 } }
I20260808 10:11:50.816771 23279 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:11:50.816809 23279 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:11:50.816857 23279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:50.817637 23279 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a357d91efc046d0952170248b0c868f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44603 } }
I20260808 10:11:50.817771 23279 leader_election.cc:302] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a357d91efc046d0952170248b0c868f; no voters:
I20260808 10:11:50.817991 23279 leader_election.cc:288] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:11:50.818114 23284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:50.818243 23284 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [term 1 LEADER]: Becoming Leader. State: Replica: 2a357d91efc046d0952170248b0c868f, State: Running, Role: LEADER
I20260808 10:11:50.818342 23284 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a357d91efc046d0952170248b0c868f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44603 } }
I20260808 10:11:50.818357 23279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:11:50.818686 23284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a357d91efc046d0952170248b0c868f. Latest consensus state: current_term: 1 leader_uuid: "2a357d91efc046d0952170248b0c868f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a357d91efc046d0952170248b0c868f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44603 } } }
I20260808 10:11:50.818764 23284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:50.818701 23287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a357d91efc046d0952170248b0c868f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a357d91efc046d0952170248b0c868f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44603 } } }
I20260808 10:11:50.818928 23287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f [sys.catalog]: This master's current role is: LEADER
W20260808 10:11:50.820461 23304 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:11:50.820576 23304 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:11:50.820698 23289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:11:50.820969 23289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:11:50.821602 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:50.809968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59989 (local address 127.0.0.1:44603)
0808 10:11:50.810175 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:50.810185 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:50.810225 (+ 40us) server_negotiation.cc:407] Connection header received
0808 10:11:50.810688 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:50.810695 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:50.810744 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:50.810906 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:50.811614 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:50.812591 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:50.813427 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:50.813852 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:50.813975 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:50.816694 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:50.816698 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:50.816698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:50.816711 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:50.819299 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:50.819822 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:50.819826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:50.819827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:50.819977 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:50.820228 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:50.820229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:50.820229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:50.820569 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:50.820752 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:50.821270 (+ 518us) server_negotiation.cc:299] Negotiation successful
0808 10:11:50.821391 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":87,"threads_started":1}
I20260808 10:11:50.822616 23289 catalog_manager.cc:1347] Generated new cluster ID: bbd42eb7b036446e9938173746ac7698
I20260808 10:11:50.822677 23289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:11:50.822690 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:11:50.833459 23289 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:11:50.834347 23289 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:11:50.839162 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:50.829645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33017 (local address 127.0.0.1:44603)
0808 10:11:50.829689 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:50.829691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:50.829895 (+ 204us) server_negotiation.cc:407] Connection header received
0808 10:11:50.830070 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:50.830071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:50.830084 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:50.830119 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:50.831116 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:50.831916 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:50.833029 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:50.833515 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:50.833599 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:50.835516 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:50.835518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:50.835519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:50.835527 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:50.837430 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:50.838033 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:50.838034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:50.838035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:50.838108 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:50.838384 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:50.838385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:50.838386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:50.838547 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:50.838638 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:50.838920 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 10:11:50.839032 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:11:50.839753 23289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a357d91efc046d0952170248b0c868f: Generated new TSK 0
I20260808 10:11:50.840018 23289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:11:50.853654 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 23209
W20260808 10:11:50.860137 23194 connection.cc:535] client connection to 127.0.0.1:44603 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44603): Connection reset by peer (error 104)
2026-08-08T10:11:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1415 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:11:50 dist-test-slave-w206 krb5kdc[23315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:11:50 dist-test-slave-w206 krb5kdc[23315](info): set up 2 sockets
Aug 08 10:11:50 dist-test-slave-w206 krb5kdc[23315](info): commencing operation
krb5kdc: starting...
W20260808 10:11:51.953018 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:11:52 dist-test-slave-w206 krb5kdc[23315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:11:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:11:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:11:52.095307 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34631
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:52.180639 23329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:52.180895 23329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:52.180920 23329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:52.182256 23329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:11:52.182322 23329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:52.182336 23329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:52.182350 23329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:11:52.182363 23329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:11:52.184044 23329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34631
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:52.184346 23329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:52.184618 23329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:52.190070 23335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:52.190061 23338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:52.190102 23329 server_base.cc:1034] running on GCE node
W20260808 10:11:52.190061 23336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:52.190659 23329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:52.191082 23329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:52.192250 23329 hybrid_clock.cc:648] HybridClock initialized: now 1786183912192230 us; error 46 us; skew 500 ppm
Aug 08 10:11:52 dist-test-slave-w206 krb5kdc[23315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183912, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:52.194873 23329 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:11:52.195364 23329 webserver.cc:460] Webserver started at http://127.0.0.1:44413/ using document root <none> and password file <none>
I20260808 10:11:52.195611 23329 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:52.195680 23329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:52.195848 23329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:52.196954 23329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "b17fedcb548d4d8290afe8e48ae03f87"
format_stamp: "Formatted at 2026-08-08 10:11:52 on dist-test-slave-w206"
server_key: "0635b4b7df332d6fe54d26e8a51653cd"
server_key_iv: "eac1f40dc3d4870d671420a7118d4e54"
server_key_version: "encryptionkey@0"
I20260808 10:11:52.197329 23329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "b17fedcb548d4d8290afe8e48ae03f87"
format_stamp: "Formatted at 2026-08-08 10:11:52 on dist-test-slave-w206"
server_key: "0635b4b7df332d6fe54d26e8a51653cd"
server_key_iv: "eac1f40dc3d4870d671420a7118d4e54"
server_key_version: "encryptionkey@0"
I20260808 10:11:52.198755 23329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:11:52.199721 23345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:52.199935 23329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:11:52.200043 23329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "b17fedcb548d4d8290afe8e48ae03f87"
format_stamp: "Formatted at 2026-08-08 10:11:52 on dist-test-slave-w206"
server_key: "0635b4b7df332d6fe54d26e8a51653cd"
server_key_iv: "eac1f40dc3d4870d671420a7118d4e54"
server_key_version: "encryptionkey@0"
I20260808 10:11:52.200134 23329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:52.210289 23329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:52.212561 23329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:52.212757 23329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:52.217223 23329 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34631
I20260808 10:11:52.217278 23400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34631 every 8 connection(s)
I20260808 10:11:52.217679 23329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:11:52.218732 23401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:52.221297 23401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87: Bootstrap starting.
I20260808 10:11:52.221668 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 23329
I20260808 10:11:52.221819 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:11:52.222062 2170 external_mini_cluster.cc:1428] Setting key 2c1f9e9df5190745cf670cc28f3c79e7
I20260808 10:11:52.222152 23401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:52.222455 23401 log.cc:824] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:52.223383 23401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87: No bootstrap required, opened a new log
I20260808 10:11:52.226017 23401 raft_consensus.cc:348] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17fedcb548d4d8290afe8e48ae03f87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34631 } }
I20260808 10:11:52.226172 23401 raft_consensus.cc:374] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:52.226195 23401 raft_consensus.cc:729] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17fedcb548d4d8290afe8e48ae03f87, State: Initialized, Role: FOLLOWER
I20260808 10:11:52.226348 23401 consensus_queue.cc:260] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17fedcb548d4d8290afe8e48ae03f87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34631 } }
I20260808 10:11:52.226426 23401 raft_consensus.cc:388] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:11:52.226450 23401 raft_consensus.cc:482] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:11:52.226523 23401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:52.227262 23401 raft_consensus.cc:504] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17fedcb548d4d8290afe8e48ae03f87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34631 } }
I20260808 10:11:52.227388 23401 leader_election.cc:302] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b17fedcb548d4d8290afe8e48ae03f87; no voters:
I20260808 10:11:52.227614 23401 leader_election.cc:288] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:11:52.227685 23406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:52.227957 23401 sys_catalog.cc:564] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:11:52.228044 23406 raft_consensus.cc:686] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [term 1 LEADER]: Becoming Leader. State: Replica: b17fedcb548d4d8290afe8e48ae03f87, State: Running, Role: LEADER
I20260808 10:11:52.228188 23406 consensus_queue.cc:237] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17fedcb548d4d8290afe8e48ae03f87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34631 } }
I20260808 10:11:52.228672 23408 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b17fedcb548d4d8290afe8e48ae03f87. Latest consensus state: current_term: 1 leader_uuid: "b17fedcb548d4d8290afe8e48ae03f87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17fedcb548d4d8290afe8e48ae03f87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34631 } } }
I20260808 10:11:52.228781 23408 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:52.228962 23405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:52.222695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55091 (local address 127.0.0.1:34631)
0808 10:11:52.222873 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:52.222883 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:52.222925 (+ 42us) server_negotiation.cc:407] Connection header received
0808 10:11:52.223651 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:52.223660 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:52.223724 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:52.223886 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:52.224126 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:52.225607 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:52.226549 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:52.227091 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:52.227218 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:52.228010 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:52.228014 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:52.228015 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:52.228263 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:52.228305 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:52.228321 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:52.228419 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:52.228541 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 10:11:52.228574 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260808 10:11:52.229102 23415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:11:52.228672 23407 sys_catalog.cc:455] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b17fedcb548d4d8290afe8e48ae03f87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b17fedcb548d4d8290afe8e48ae03f87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34631 } } }
I20260808 10:11:52.229164 23407 sys_catalog.cc:458] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87 [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:52.229666 23415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:11:52.230409 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:11:52.235590 23415 catalog_manager.cc:1347] Generated new cluster ID: 65b2c4011a6d4c609092faf04191885a
I20260808 10:11:52.235666 23415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:11:52.238736 23405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:52.232203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43771 (local address 127.0.0.1:34631)
0808 10:11:52.234773 (+ 2570us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:52.234775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:52.234786 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:11:52.234813 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:52.234814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:52.234827 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:52.234892 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:52.235362 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:52.236142 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:52.237206 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:52.237534 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:52.237584 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:52.238470 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:52.238471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:52.238472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:52.238477 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:52.238494 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:52.238498 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:52.238558 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:52.238638 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 10:11:52.238649 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2542}
I20260808 10:11:52.253788 23415 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:11:52.254637 23415 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:11:52.261183 23415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b17fedcb548d4d8290afe8e48ae03f87: Generated new TSK 0
I20260808 10:11:52.261475 23415 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:11:52.265844 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 23329
W20260808 10:11:52.272334 23313 connection.cc:535] client connection to 127.0.0.1:34631 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34631): Connection reset by peer (error 104)
2026-08-08T10:11:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:11:52 dist-test-slave-w206 krb5kdc[23432](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:11:52 dist-test-slave-w206 krb5kdc[23432](info): set up 2 sockets
Aug 08 10:11:52 dist-test-slave-w206 krb5kdc[23432](info): commencing operation
krb5kdc: starting...
W20260808 10:11:53.352093 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:11:53 dist-test-slave-w206 krb5kdc[23432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:11:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:11:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:11:53.494478 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36293
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:53.584868 23448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:53.585139 23448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:53.585178 23448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:53.586432 23448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:11:53.586510 23448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:53.586536 23448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:53.586566 23448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:11:53.586598 23448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:11:53.588203 23448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36293
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:53.588505 23448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:53.588744 23448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:53.593957 23453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:53.593998 23448 server_base.cc:1034] running on GCE node
W20260808 10:11:53.593937 23454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:53.593937 23456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:53.594384 23448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:53.594707 23448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:53.595885 23448 hybrid_clock.cc:648] HybridClock initialized: now 1786183913595926 us; error 96 us; skew 500 ppm
Aug 08 10:11:53 dist-test-slave-w206 krb5kdc[23432](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183913, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:53.598448 23448 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:11:53.598927 23448 webserver.cc:460] Webserver started at http://127.0.0.1:37115/ using document root <none> and password file <none>
I20260808 10:11:53.599190 23448 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:53.599263 23448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:53.599426 23448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:53.600605 23448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "4a9fbe5cb06b476684aa8e2246b5b0d7"
format_stamp: "Formatted at 2026-08-08 10:11:53 on dist-test-slave-w206"
server_key: "40c0799085ef4b1ec3368ea8ffda8fa2"
server_key_iv: "49fcaa4581616427539bafd5e728b843"
server_key_version: "encryptionkey@0"
I20260808 10:11:53.600978 23448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "4a9fbe5cb06b476684aa8e2246b5b0d7"
format_stamp: "Formatted at 2026-08-08 10:11:53 on dist-test-slave-w206"
server_key: "40c0799085ef4b1ec3368ea8ffda8fa2"
server_key_iv: "49fcaa4581616427539bafd5e728b843"
server_key_version: "encryptionkey@0"
I20260808 10:11:53.602381 23448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:11:53.603184 23463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:53.603363 23448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:53.603472 23448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "4a9fbe5cb06b476684aa8e2246b5b0d7"
format_stamp: "Formatted at 2026-08-08 10:11:53 on dist-test-slave-w206"
server_key: "40c0799085ef4b1ec3368ea8ffda8fa2"
server_key_iv: "49fcaa4581616427539bafd5e728b843"
server_key_version: "encryptionkey@0"
I20260808 10:11:53.603550 23448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:53.606890 23448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:53.608886 23448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:53.609046 23448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:53.613245 23448 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36293
I20260808 10:11:53.613286 23518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36293 every 8 connection(s)
I20260808 10:11:53.613610 23448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:11:53.614629 23520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:53.616969 23520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7: Bootstrap starting.
I20260808 10:11:53.617563 23520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:53.617862 23520 log.cc:824] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:53.618564 23520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7: No bootstrap required, opened a new log
I20260808 10:11:53.620266 23520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a9fbe5cb06b476684aa8e2246b5b0d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36293 } }
I20260808 10:11:53.620414 23520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:53.620437 23520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a9fbe5cb06b476684aa8e2246b5b0d7, State: Initialized, Role: FOLLOWER
I20260808 10:11:53.620576 23520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a9fbe5cb06b476684aa8e2246b5b0d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36293 } }
I20260808 10:11:53.620659 23520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:11:53.620711 23520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:11:53.620726 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 23448
I20260808 10:11:53.620792 23520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:53.620877 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:11:53.621106 2170 external_mini_cluster.cc:1428] Setting key 6aea53baafc56134e91ca482d5f0a588
I20260808 10:11:53.621434 23520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a9fbe5cb06b476684aa8e2246b5b0d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36293 } }
I20260808 10:11:53.621558 23520 leader_election.cc:302] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a9fbe5cb06b476684aa8e2246b5b0d7; no voters:
I20260808 10:11:53.621793 23520 leader_election.cc:288] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:11:53.621819 23525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:53.621997 23525 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [term 1 LEADER]: Becoming Leader. State: Replica: 4a9fbe5cb06b476684aa8e2246b5b0d7, State: Running, Role: LEADER
I20260808 10:11:53.622128 23520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:11:53.622128 23525 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a9fbe5cb06b476684aa8e2246b5b0d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36293 } }
I20260808 10:11:53.622910 23526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a9fbe5cb06b476684aa8e2246b5b0d7. Latest consensus state: current_term: 1 leader_uuid: "4a9fbe5cb06b476684aa8e2246b5b0d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a9fbe5cb06b476684aa8e2246b5b0d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36293 } } }
I20260808 10:11:53.622900 23527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a9fbe5cb06b476684aa8e2246b5b0d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a9fbe5cb06b476684aa8e2246b5b0d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36293 } } }
I20260808 10:11:53.623126 23527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:53.623739 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:53.621607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39549 (local address 127.0.0.1:36293)
0808 10:11:53.621798 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:53.621810 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:53.621855 (+ 45us) server_negotiation.cc:407] Connection header received
0808 10:11:53.621989 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:53.621999 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:53.622035 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:53.622538 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:53.623092 (+ 554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:53.623096 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:53.623096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:53.623101 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:53.623154 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:53.623172 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:53.623488 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:53.623522 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 10:11:53.623567 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260808 10:11:53.623808 23533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:11:53.624262 23526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:53.624593 23533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:11:53.624691 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:11:53.626297 23533 catalog_manager.cc:1347] Generated new cluster ID: 155eb65eb99745fcadcb1cae8e874354
I20260808 10:11:53.626358 23533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:11:53.627535 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:53.626436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42899 (local address 127.0.0.1:36293)
0808 10:11:53.626494 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:53.626496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:53.626505 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:11:53.626528 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:53.626528 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:53.626533 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:53.626604 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:53.627297 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:53.627299 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:53.627299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:53.627303 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:53.627319 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:53.627324 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:53.627388 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:53.627458 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 10:11:53.627471 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 10:11:53.636915 23533 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:11:53.637141 23533 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:11:53.642371 23533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a9fbe5cb06b476684aa8e2246b5b0d7: Generated new TSK 0
I20260808 10:11:53.642608 23533 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:11:53.646591 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 23448
2026-08-08T10:11:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1380 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:11:53 dist-test-slave-w206 krb5kdc[23551](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:11:53 dist-test-slave-w206 krb5kdc[23551](info): set up 2 sockets
Aug 08 10:11:53 dist-test-slave-w206 krb5kdc[23551](info): commencing operation
krb5kdc: starting...
W20260808 10:11:54.744609 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:11:54 dist-test-slave-w206 krb5kdc[23551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:11:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:11:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:11:54.883311 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36515
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:54.969281 23566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:54.969507 23566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:54.969532 23566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:54.970711 23566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:11:54.970758 23566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:54.970773 23566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:54.970786 23566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:11:54.970799 23566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:11:54.972306 23566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:54.972545 23566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:54.972736 23566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:54.978384 23571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:54.978375 23574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:54.978370 23572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:54.978394 23566 server_base.cc:1034] running on GCE node
I20260808 10:11:54.978842 23566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:54.979245 23566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:54.980482 23566 hybrid_clock.cc:648] HybridClock initialized: now 1786183914980468 us; error 31 us; skew 500 ppm
Aug 08 10:11:54 dist-test-slave-w206 krb5kdc[23551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183914, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:54.983759 23566 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:11:54.984345 23566 webserver.cc:460] Webserver started at http://127.0.0.1:45265/ using document root <none> and password file <none>
I20260808 10:11:54.984625 23566 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:54.984694 23566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:54.984856 23566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:54.986040 23566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "f3d0c838d79b458fb0f9372907542b2b"
format_stamp: "Formatted at 2026-08-08 10:11:54 on dist-test-slave-w206"
server_key: "6938772e298bcb20e9ace0bdc62ec31c"
server_key_iv: "6db8df2bf4ab7febbd02d8e5b9043c43"
server_key_version: "encryptionkey@0"
I20260808 10:11:54.986430 23566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "f3d0c838d79b458fb0f9372907542b2b"
format_stamp: "Formatted at 2026-08-08 10:11:54 on dist-test-slave-w206"
server_key: "6938772e298bcb20e9ace0bdc62ec31c"
server_key_iv: "6db8df2bf4ab7febbd02d8e5b9043c43"
server_key_version: "encryptionkey@0"
I20260808 10:11:54.987993 23566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:11:54.988878 23582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:54.989082 23566 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:11:54.989212 23566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "f3d0c838d79b458fb0f9372907542b2b"
format_stamp: "Formatted at 2026-08-08 10:11:54 on dist-test-slave-w206"
server_key: "6938772e298bcb20e9ace0bdc62ec31c"
server_key_iv: "6db8df2bf4ab7febbd02d8e5b9043c43"
server_key_version: "encryptionkey@0"
I20260808 10:11:54.989331 23566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:55.006650 23566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:55.009353 23566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:55.009531 23566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:55.014052 23566 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36515
I20260808 10:11:55.014158 23636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36515 every 8 connection(s)
I20260808 10:11:55.014505 23566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:11:55.015697 23637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:55.018123 23637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b: Bootstrap starting.
I20260808 10:11:55.018747 23637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:55.019100 23637 log.cc:824] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:55.019522 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 23566
I20260808 10:11:55.019658 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:11:55.019870 2170 external_mini_cluster.cc:1428] Setting key 43125d0403a1e10ac386ca97ec04e936
I20260808 10:11:55.019979 23637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b: No bootstrap required, opened a new log
I20260808 10:11:55.022552 23637 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3d0c838d79b458fb0f9372907542b2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36515 } }
I20260808 10:11:55.022710 23637 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:55.022734 23637 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3d0c838d79b458fb0f9372907542b2b, State: Initialized, Role: FOLLOWER
I20260808 10:11:55.022892 23637 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3d0c838d79b458fb0f9372907542b2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36515 } }
I20260808 10:11:55.023031 23637 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:11:55.023077 23637 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:11:55.023132 23637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:55.023829 23637 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3d0c838d79b458fb0f9372907542b2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36515 } }
I20260808 10:11:55.024019 23637 leader_election.cc:302] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3d0c838d79b458fb0f9372907542b2b; no voters:
I20260808 10:11:55.024199 23637 leader_election.cc:288] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:11:55.024261 23642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:55.024447 23642 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [term 1 LEADER]: Becoming Leader. State: Replica: f3d0c838d79b458fb0f9372907542b2b, State: Running, Role: LEADER
I20260808 10:11:55.024508 23637 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:11:55.024585 23642 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3d0c838d79b458fb0f9372907542b2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36515 } }
I20260808 10:11:55.024986 23642 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3d0c838d79b458fb0f9372907542b2b. Latest consensus state: current_term: 1 leader_uuid: "f3d0c838d79b458fb0f9372907542b2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3d0c838d79b458fb0f9372907542b2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36515 } } }
I20260808 10:11:55.024992 23645 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3d0c838d79b458fb0f9372907542b2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3d0c838d79b458fb0f9372907542b2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36515 } } }
I20260808 10:11:55.025136 23642 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:55.025144 23645 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:55.025904 23652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:11:55.026672 23652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:11:55.028275 23652 catalog_manager.cc:1347] Generated new cluster ID: 75dcf9cce73a4463b3d091744d51c7b4
I20260808 10:11:55.028337 23652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:11:55 dist-test-slave-w206 krb5kdc[23551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:11:55.038300 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:55.020441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56875 (local address 127.0.0.1:36515)
0808 10:11:55.020631 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:55.020639 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:55.020665 (+ 26us) server_negotiation.cc:407] Connection header received
0808 10:11:55.021363 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:55.021370 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:55.021419 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:55.021530 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:55.022386 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:55.023826 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:55.025068 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:55.025778 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:55.025871 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:55.033932 (+ 8061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:55.033938 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:55.033938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:55.033953 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:55.036143 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:55.036625 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:55.036628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:55.036629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:55.036742 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:55.036960 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:55.036961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:55.036961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:55.037287 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:55.037438 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:55.037901 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 10:11:55.038040 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260808 10:11:55.039361 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:11:55.048301 23652 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:11:55.048468 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:55.040958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35047 (local address 127.0.0.1:36515)
0808 10:11:55.040996 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:55.040998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:55.041060 (+ 62us) server_negotiation.cc:407] Connection header received
0808 10:11:55.041119 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:55.041120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:55.041130 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:55.041198 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:55.041836 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:55.042603 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:55.043632 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:55.043967 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:55.044042 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:55.045425 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:55.045426 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:55.045427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:55.045435 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:55.047140 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:55.047544 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:55.047545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:55.047546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:55.047611 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:55.047857 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:55.047859 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:55.047859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:55.047978 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:55.048084 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:55.048284 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 10:11:55.048372 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:11:55.049552 23652 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:11:55.058049 23652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3d0c838d79b458fb0f9372907542b2b: Generated new TSK 0
I20260808 10:11:55.058286 23652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:11:55.064105 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 23566
2026-08-08T10:11:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1417 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:11:55 dist-test-slave-w206 krb5kdc[23673](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:11:55 dist-test-slave-w206 krb5kdc[23673](info): set up 2 sockets
Aug 08 10:11:55 dist-test-slave-w206 krb5kdc[23673](info): commencing operation
krb5kdc: starting...
W20260808 10:11:56.141144 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:11:56 dist-test-slave-w206 krb5kdc[23673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:11:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:11:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:11:56.286870 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46005
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:56.373296 23686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:56.373588 23686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:56.373633 23686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:56.375080 23686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:11:56.375133 23686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:56.375149 23686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:56.375162 23686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:11:56.375174 23686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:11:56.376760 23686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:56.376983 23686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:56.377170 23686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:56.382742 23692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:56.382813 23691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:56.382745 23694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:56.382764 23686 server_base.cc:1034] running on GCE node
I20260808 10:11:56.383217 23686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:56.383612 23686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:56.384799 23686 hybrid_clock.cc:648] HybridClock initialized: now 1786183916384801 us; error 49 us; skew 500 ppm
Aug 08 10:11:56 dist-test-slave-w206 krb5kdc[23673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183916, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:56.394739 23686 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:11:56.395349 23686 webserver.cc:460] Webserver started at http://127.0.0.1:40495/ using document root <none> and password file <none>
I20260808 10:11:56.395643 23686 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:56.395715 23686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:56.395874 23686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:56.397034 23686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "e69677caad5f445fba9e85034ff0e3ab"
format_stamp: "Formatted at 2026-08-08 10:11:56 on dist-test-slave-w206"
server_key: "0ead09592470e8cf9da091de397bc024"
server_key_iv: "cbfcd84e12657ed575fbd1618abf5fec"
server_key_version: "encryptionkey@0"
I20260808 10:11:56.397413 23686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "e69677caad5f445fba9e85034ff0e3ab"
format_stamp: "Formatted at 2026-08-08 10:11:56 on dist-test-slave-w206"
server_key: "0ead09592470e8cf9da091de397bc024"
server_key_iv: "cbfcd84e12657ed575fbd1618abf5fec"
server_key_version: "encryptionkey@0"
I20260808 10:11:56.398937 23686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:11:56.399956 23701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:56.400180 23686 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:56.400300 23686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "e69677caad5f445fba9e85034ff0e3ab"
format_stamp: "Formatted at 2026-08-08 10:11:56 on dist-test-slave-w206"
server_key: "0ead09592470e8cf9da091de397bc024"
server_key_iv: "cbfcd84e12657ed575fbd1618abf5fec"
server_key_version: "encryptionkey@0"
I20260808 10:11:56.400413 23686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:56.427886 23686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:56.430212 23686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:56.430460 23686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:56.435122 23686 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46005
I20260808 10:11:56.435166 23754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46005 every 8 connection(s)
I20260808 10:11:56.435551 23686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:11:56.436618 23755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:56.439123 23755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab: Bootstrap starting.
I20260808 10:11:56.439764 23755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:56.440080 23755 log.cc:824] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:56.440850 23755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab: No bootstrap required, opened a new log
I20260808 10:11:56.442570 23755 raft_consensus.cc:348] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e69677caad5f445fba9e85034ff0e3ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46005 } }
I20260808 10:11:56.442718 23755 raft_consensus.cc:374] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:56.442742 23755 raft_consensus.cc:729] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69677caad5f445fba9e85034ff0e3ab, State: Initialized, Role: FOLLOWER
I20260808 10:11:56.442919 23755 consensus_queue.cc:260] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e69677caad5f445fba9e85034ff0e3ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46005 } }
I20260808 10:11:56.443058 23755 raft_consensus.cc:388] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:11:56.443099 23755 raft_consensus.cc:482] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:11:56.443166 23755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:56.443672 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 23686
I20260808 10:11:56.443810 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:11:56.444043 2170 external_mini_cluster.cc:1428] Setting key 248723730e5ac2e5b78abbf41351ea0e
I20260808 10:11:56.444022 23755 raft_consensus.cc:504] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e69677caad5f445fba9e85034ff0e3ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46005 } }
I20260808 10:11:56.444129 23755 leader_election.cc:302] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e69677caad5f445fba9e85034ff0e3ab; no voters:
I20260808 10:11:56.444370 23755 leader_election.cc:288] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:11:56.444419 23758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:56.444602 23758 raft_consensus.cc:686] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [term 1 LEADER]: Becoming Leader. State: Replica: e69677caad5f445fba9e85034ff0e3ab, State: Running, Role: LEADER
I20260808 10:11:56.444661 23755 sys_catalog.cc:564] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:11:56.444950 23758 consensus_queue.cc:237] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e69677caad5f445fba9e85034ff0e3ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46005 } }
I20260808 10:11:56.445670 23759 sys_catalog.cc:455] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e69677caad5f445fba9e85034ff0e3ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e69677caad5f445fba9e85034ff0e3ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46005 } } }
I20260808 10:11:56.445796 23759 sys_catalog.cc:458] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:56.446732 23760 sys_catalog.cc:455] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader e69677caad5f445fba9e85034ff0e3ab. Latest consensus state: current_term: 1 leader_uuid: "e69677caad5f445fba9e85034ff0e3ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e69677caad5f445fba9e85034ff0e3ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46005 } } }
I20260808 10:11:56.446820 23760 sys_catalog.cc:458] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab [sys.catalog]: This master's current role is: LEADER
W20260808 10:11:56.447034 23776 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:11:56.447091 23776 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:11:56.447134 23769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:11:56.447281 23769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:11:56.448949 23769 catalog_manager.cc:1347] Generated new cluster ID: a7e03dd942fe4cd592449a5538201ecb
I20260808 10:11:56.449003 23769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:11:56.451131 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:56.444790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35745 (local address 127.0.0.1:46005)
0808 10:11:56.445014 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:56.445025 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:56.445074 (+ 49us) server_negotiation.cc:407] Connection header received
0808 10:11:56.445194 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:56.445201 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:56.445261 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:56.445420 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:56.446264 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:56.447732 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:56.448770 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:56.449354 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:56.449480 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:56.450261 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:56.450265 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:56.450266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:56.450488 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:56.450524 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:56.450537 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:56.450628 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:56.450708 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 10:11:56.450876 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":79,"threads_started":1}
I20260808 10:11:56.452104 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:11:56.456691 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:56.453603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45279 (local address 127.0.0.1:46005)
0808 10:11:56.453644 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:56.453646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:56.453777 (+ 131us) server_negotiation.cc:407] Connection header received
0808 10:11:56.453835 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:56.453837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:56.453848 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:56.453895 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:56.454048 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:56.454541 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:56.455407 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:56.455624 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:56.455717 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:56.456421 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:56.456422 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:56.456423 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:56.456428 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:56.456443 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:56.456448 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:56.456497 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:56.456555 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 10:11:56.456578 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:11:56.461387 23769 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:11:56.462141 23769 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:11:56.468636 23769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e69677caad5f445fba9e85034ff0e3ab: Generated new TSK 0
I20260808 10:11:56.468889 23769 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:11:56.471800 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 23686
2026-08-08T10:11:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1408 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:11:56 dist-test-slave-w206 krb5kdc[23786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:11:56 dist-test-slave-w206 krb5kdc[23786](info): set up 2 sockets
Aug 08 10:11:56 dist-test-slave-w206 krb5kdc[23786](info): commencing operation
krb5kdc: starting...
W20260808 10:11:57.552594 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:11:57 dist-test-slave-w206 krb5kdc[23786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:11:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:11:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:11:57.691365 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40541
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:57.778888 23800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:57.779255 23800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:57.779294 23800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:57.780526 23800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:11:57.780570 23800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:57.780584 23800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:57.780597 23800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:11:57.780611 23800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:11:57.782166 23800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:57.782398 23800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:57.782594 23800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:57.788509 23810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:57.788511 23806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:57.788549 23800 server_base.cc:1034] running on GCE node
W20260808 10:11:57.788509 23805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:57.789057 23800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:57.789438 23800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:57.790634 23800 hybrid_clock.cc:648] HybridClock initialized: now 1786183917790616 us; error 30 us; skew 500 ppm
Aug 08 10:11:57 dist-test-slave-w206 krb5kdc[23786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183917, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:57.793812 23800 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:11:57.794396 23800 webserver.cc:460] Webserver started at http://127.0.0.1:43613/ using document root <none> and password file <none>
I20260808 10:11:57.794687 23800 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:57.794763 23800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:57.794927 23800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:57.796108 23800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "fb7f3c940e394597b18f6f811127347b"
format_stamp: "Formatted at 2026-08-08 10:11:57 on dist-test-slave-w206"
server_key: "10903d6f30dd45be8140957ae070ce08"
server_key_iv: "447342d4e9f73f488101877786b9515e"
server_key_version: "encryptionkey@0"
I20260808 10:11:57.796510 23800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "fb7f3c940e394597b18f6f811127347b"
format_stamp: "Formatted at 2026-08-08 10:11:57 on dist-test-slave-w206"
server_key: "10903d6f30dd45be8140957ae070ce08"
server_key_iv: "447342d4e9f73f488101877786b9515e"
server_key_version: "encryptionkey@0"
I20260808 10:11:57.798051 23800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 10:11:57.799098 23818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:57.799327 23800 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:57.799451 23800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "fb7f3c940e394597b18f6f811127347b"
format_stamp: "Formatted at 2026-08-08 10:11:57 on dist-test-slave-w206"
server_key: "10903d6f30dd45be8140957ae070ce08"
server_key_iv: "447342d4e9f73f488101877786b9515e"
server_key_version: "encryptionkey@0"
I20260808 10:11:57.799537 23800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:57.803532 23800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:57.806016 23800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:57.806304 23800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:57.811097 23800 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40541
I20260808 10:11:57.811134 23872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40541 every 8 connection(s)
I20260808 10:11:57.811537 23800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:11:57.812722 23873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:57.815294 23873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b: Bootstrap starting.
I20260808 10:11:57.815927 23873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:57.816260 23873 log.cc:824] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:57.817041 23873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b: No bootstrap required, opened a new log
I20260808 10:11:57.817823 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 23800
I20260808 10:11:57.817976 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:11:57.818187 2170 external_mini_cluster.cc:1428] Setting key 3aba17451af76f94ab6abf50ca5ae422
I20260808 10:11:57.819355 23873 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7f3c940e394597b18f6f811127347b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40541 } }
I20260808 10:11:57.819551 23873 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:57.819594 23873 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb7f3c940e394597b18f6f811127347b, State: Initialized, Role: FOLLOWER
I20260808 10:11:57.819814 23873 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7f3c940e394597b18f6f811127347b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40541 } }
I20260808 10:11:57.819926 23873 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:11:57.819952 23873 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:11:57.820045 23873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:57.820778 23877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:57.818817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43921 (local address 127.0.0.1:40541)
0808 10:11:57.819055 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:57.819063 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:57.819090 (+ 27us) server_negotiation.cc:407] Connection header received
0808 10:11:57.819329 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:57.819335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:57.819363 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:57.819483 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:57.820295 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:57.820299 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:57.820299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:57.820303 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:57.820344 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:57.820356 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:57.820412 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:57.820507 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 10:11:57.820565 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":111,"threads_started":1}
I20260808 10:11:57.821312 23873 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7f3c940e394597b18f6f811127347b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40541 } }
I20260808 10:11:57.821432 23873 leader_election.cc:302] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb7f3c940e394597b18f6f811127347b; no voters:
I20260808 10:11:57.821596 23873 leader_election.cc:288] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:11:57.821626 23878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:57.821786 23878 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [term 1 LEADER]: Becoming Leader. State: Replica: fb7f3c940e394597b18f6f811127347b, State: Running, Role: LEADER
I20260808 10:11:57.821846 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:11:57.821871 23873 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:11:57.821914 23878 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7f3c940e394597b18f6f811127347b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40541 } }
I20260808 10:11:57.822307 23878 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb7f3c940e394597b18f6f811127347b. Latest consensus state: current_term: 1 leader_uuid: "fb7f3c940e394597b18f6f811127347b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7f3c940e394597b18f6f811127347b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40541 } } }
I20260808 10:11:57.822319 23879 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb7f3c940e394597b18f6f811127347b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7f3c940e394597b18f6f811127347b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40541 } } }
I20260808 10:11:57.822568 23879 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:57.822485 23878 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:57.822827 23888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:11:57.823488 23888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:11:57.825242 23877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:57.823741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49747 (local address 127.0.0.1:40541)
0808 10:11:57.823818 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:57.823820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:57.824047 (+ 227us) server_negotiation.cc:407] Connection header received
0808 10:11:57.824123 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:57.824124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:57.824129 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:57.824174 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:57.825021 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:57.825022 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:57.825023 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:11:57.825027 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:57.825042 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:11:57.825046 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:57.825098 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:57.825153 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 10:11:57.825168 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:11:57.825390 23888 catalog_manager.cc:1347] Generated new cluster ID: 1bfb75b4411246be9b1aa62b468b1e78
I20260808 10:11:57.825444 23888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:11:57.835304 23888 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:11:57.835572 23888 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:11:57.858126 23888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb7f3c940e394597b18f6f811127347b: Generated new TSK 0
I20260808 10:11:57.858372 23888 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:11:57.868955 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 23800
2026-08-08T10:11:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1396 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:11:57 dist-test-slave-w206 krb5kdc[23905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:11:57 dist-test-slave-w206 krb5kdc[23905](info): set up 2 sockets
Aug 08 10:11:57 dist-test-slave-w206 krb5kdc[23905](info): commencing operation
krb5kdc: starting...
W20260808 10:11:58.960637 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:11:59 dist-test-slave-w206 krb5kdc[23905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:11:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:11:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:11:59.102468 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32983
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:11:59.189860 23919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:11:59.190105 23919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:11:59.190131 23919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:11:59.191404 23919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:11:59.191452 23919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:11:59.191466 23919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:11:59.191479 23919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:11:59.191493 23919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:11:59.193077 23919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:11:59.193318 23919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:11:59.193521 23919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:11:59.199426 23927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:59.199412 23924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:11:59.199412 23925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:11:59.200526 23919 server_base.cc:1034] running on GCE node
I20260808 10:11:59.200876 23919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:11:59.201259 23919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:11:59.202533 23919 hybrid_clock.cc:648] HybridClock initialized: now 1786183919202451 us; error 52 us; skew 500 ppm
Aug 08 10:11:59 dist-test-slave-w206 krb5kdc[23905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:11:59.205678 23919 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:11:59.206292 23919 webserver.cc:460] Webserver started at http://127.0.0.1:37429/ using document root <none> and password file <none>
I20260808 10:11:59.206552 23919 fs_manager.cc:359] Metadata directory not provided
I20260808 10:11:59.206630 23919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:11:59.206799 23919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:11:59.208033 23919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "2685643ccab24871918e60157d8bc27a"
format_stamp: "Formatted at 2026-08-08 10:11:59 on dist-test-slave-w206"
server_key: "e7d2d7a9869b69844f620ee2a99adbcd"
server_key_iv: "b2276ba6c953e80f2dfdd612254f31af"
server_key_version: "encryptionkey@0"
I20260808 10:11:59.208541 23919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "2685643ccab24871918e60157d8bc27a"
format_stamp: "Formatted at 2026-08-08 10:11:59 on dist-test-slave-w206"
server_key: "e7d2d7a9869b69844f620ee2a99adbcd"
server_key_iv: "b2276ba6c953e80f2dfdd612254f31af"
server_key_version: "encryptionkey@0"
I20260808 10:11:59.210093 23919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:11:59.211103 23934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:11:59.211366 23919 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:11:59.211477 23919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "2685643ccab24871918e60157d8bc27a"
format_stamp: "Formatted at 2026-08-08 10:11:59 on dist-test-slave-w206"
server_key: "e7d2d7a9869b69844f620ee2a99adbcd"
server_key_iv: "b2276ba6c953e80f2dfdd612254f31af"
server_key_version: "encryptionkey@0"
I20260808 10:11:59.211565 23919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:11:59.232309 23919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:11:59.235249 23919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:11:59.235455 23919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:11:59.241048 23919 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32983
I20260808 10:11:59.241215 23989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32983 every 8 connection(s)
I20260808 10:11:59.241576 23919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:11:59.242645 23990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:11:59.245502 23990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a: Bootstrap starting.
I20260808 10:11:59.246240 23990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a: Neither blocks nor log segments found. Creating new log.
I20260808 10:11:59.246591 23990 log.cc:824] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:11:59.247505 23990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a: No bootstrap required, opened a new log
I20260808 10:11:59.249105 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 23919
I20260808 10:11:59.249212 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:11:59.249481 2170 external_mini_cluster.cc:1428] Setting key cdf8fd83acb143ae654824c883b0f1e7
I20260808 10:11:59.249312 23990 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2685643ccab24871918e60157d8bc27a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32983 } }
I20260808 10:11:59.249540 23990 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:11:59.249598 23990 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2685643ccab24871918e60157d8bc27a, State: Initialized, Role: FOLLOWER
I20260808 10:11:59.249778 23990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2685643ccab24871918e60157d8bc27a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32983 } }
I20260808 10:11:59.249868 23990 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:11:59.249902 23990 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:11:59.249979 23990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:11:59.251304 23990 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2685643ccab24871918e60157d8bc27a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32983 } }
I20260808 10:11:59.251479 23990 leader_election.cc:302] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2685643ccab24871918e60157d8bc27a; no voters:
I20260808 10:11:59.251686 23990 leader_election.cc:288] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:11:59.251719 23996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:11:59.251834 23996 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [term 1 LEADER]: Becoming Leader. State: Replica: 2685643ccab24871918e60157d8bc27a, State: Running, Role: LEADER
I20260808 10:11:59.251989 23990 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:11:59.252004 23996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2685643ccab24871918e60157d8bc27a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32983 } }
I20260808 10:11:59.252564 23998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2685643ccab24871918e60157d8bc27a. Latest consensus state: current_term: 1 leader_uuid: "2685643ccab24871918e60157d8bc27a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2685643ccab24871918e60157d8bc27a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32983 } } }
I20260808 10:11:59.252662 23998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:59.252928 23997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2685643ccab24871918e60157d8bc27a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2685643ccab24871918e60157d8bc27a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32983 } } }
I20260808 10:11:59.253041 23997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a [sys.catalog]: This master's current role is: LEADER
I20260808 10:11:59.253350 24004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:11:59.254004 24004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:11:59.255932 24004 catalog_manager.cc:1347] Generated new cluster ID: 06a760fcca0e42a6a9397f7498da1ad2
I20260808 10:11:59.255995 24004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:11:59 dist-test-slave-w206 krb5kdc[23905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:11:59.270488 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:59.250145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:60161 (local address 127.0.0.1:32983)
0808 10:11:59.250397 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:59.250408 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:59.250478 (+ 70us) server_negotiation.cc:407] Connection header received
0808 10:11:59.250632 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:59.250640 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:59.250734 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:59.250918 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:59.251883 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:59.253413 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:59.254613 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:59.255160 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:59.255296 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:59.265922 (+ 10626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:59.265929 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:59.265929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:59.265947 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:59.268379 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:59.268985 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:59.268989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:59.268990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:59.269115 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:59.269422 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:59.269423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:59.269424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:59.269756 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:59.269893 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:59.270101 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 10:11:59.270209 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":124,"threads_started":1}
I20260808 10:11:59.271612 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:11:59.281451 24004 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:11:59.282219 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:11:59.273176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:55063 (local address 127.0.0.1:32983)
0808 10:11:59.273219 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 10:11:59.273221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:11:59.273359 (+ 138us) server_negotiation.cc:407] Connection header received
0808 10:11:59.273476 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:11:59.273477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:11:59.273501 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:11:59.273561 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:11:59.274344 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:59.275145 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:59.276333 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:11:59.277187 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:11:59.277263 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:11:59.279057 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:11:59.279059 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:11:59.279059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:11:59.279067 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:11:59.280918 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:59.281384 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:59.281386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:59.281386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:59.281454 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:11:59.281661 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:11:59.281662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:11:59.281663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:11:59.281776 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:11:59.281898 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 10:11:59.282046 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 10:11:59.282111 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:11:59.282377 24004 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:11:59.292093 24004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2685643ccab24871918e60157d8bc27a: Generated new TSK 0
I20260808 10:11:59.292369 24004 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:11:59.298290 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 23919
W20260808 10:11:59.305323 23903 connection.cc:535] client connection to 127.0.0.1:32983 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32983): Connection reset by peer (error 104)
2026-08-08T10:11:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1429 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:11:59 dist-test-slave-w206 krb5kdc[24022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:11:59 dist-test-slave-w206 krb5kdc[24022](info): set up 2 sockets
Aug 08 10:11:59 dist-test-slave-w206 krb5kdc[24022](info): commencing operation
krb5kdc: starting...
W20260808 10:12:00.399289 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:12:00 dist-test-slave-w206 krb5kdc[24022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:12:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:12:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:12:00.534454 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42111
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:12:00.619977 24036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:00.620201 24036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:00.620229 24036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:00.621402 24036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:00.621444 24036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:00.621459 24036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:00.621474 24036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:00.621485 24036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:00.623055 24036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:00.623301 24036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:00.623504 24036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:12:00.628890 24041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:00.628890 24042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:00.628944 24044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:00.628999 24036 server_base.cc:1034] running on GCE node
I20260808 10:12:00.629424 24036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:00.629746 24036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:00.630939 24036 hybrid_clock.cc:648] HybridClock initialized: now 1786183920630910 us; error 49 us; skew 500 ppm
Aug 08 10:12:00 dist-test-slave-w206 krb5kdc[24022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183920, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:00.633929 24036 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:12:00.634550 24036 webserver.cc:460] Webserver started at http://127.0.0.1:35847/ using document root <none> and password file <none>
I20260808 10:12:00.634832 24036 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:00.634914 24036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:00.635113 24036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:00.636341 24036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "a0f9626d7e9242b5b6562f9fe3649774"
format_stamp: "Formatted at 2026-08-08 10:12:00 on dist-test-slave-w206"
server_key: "5f4929344e12bb3e00085e97614856aa"
server_key_iv: "7c03316d6aaf3881b1d716e2b8f3ad60"
server_key_version: "encryptionkey@0"
I20260808 10:12:00.636747 24036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "a0f9626d7e9242b5b6562f9fe3649774"
format_stamp: "Formatted at 2026-08-08 10:12:00 on dist-test-slave-w206"
server_key: "5f4929344e12bb3e00085e97614856aa"
server_key_iv: "7c03316d6aaf3881b1d716e2b8f3ad60"
server_key_version: "encryptionkey@0"
I20260808 10:12:00.638356 24036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:12:00.639348 24051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:00.639539 24036 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:12:00.639662 24036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "a0f9626d7e9242b5b6562f9fe3649774"
format_stamp: "Formatted at 2026-08-08 10:12:00 on dist-test-slave-w206"
server_key: "5f4929344e12bb3e00085e97614856aa"
server_key_iv: "7c03316d6aaf3881b1d716e2b8f3ad60"
server_key_version: "encryptionkey@0"
I20260808 10:12:00.639756 24036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:00.668471 24036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:00.670748 24036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:00.670934 24036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:00.675060 24036 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42111
I20260808 10:12:00.675078 24104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42111 every 8 connection(s)
I20260808 10:12:00.675458 24036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:12:00.676487 24105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:00.678781 24105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774: Bootstrap starting.
I20260808 10:12:00.679394 24105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:00.679657 24105 log.cc:824] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:00.680411 24105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774: No bootstrap required, opened a new log
I20260808 10:12:00.681010 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 24036
I20260808 10:12:00.681143 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:12:00.681360 2170 external_mini_cluster.cc:1428] Setting key 7563031e643891142a2274bd4b627c80
I20260808 10:12:00.682590 24105 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f9626d7e9242b5b6562f9fe3649774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } }
I20260808 10:12:00.682765 24105 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:12:00.682806 24105 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0f9626d7e9242b5b6562f9fe3649774, State: Initialized, Role: FOLLOWER
I20260808 10:12:00.683013 24105 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f9626d7e9242b5b6562f9fe3649774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } }
I20260808 10:12:00.683143 24105 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:12:00.683199 24105 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:12:00.683259 24105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:00.683826 24105 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f9626d7e9242b5b6562f9fe3649774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } }
I20260808 10:12:00.683914 24105 leader_election.cc:302] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0f9626d7e9242b5b6562f9fe3649774; no voters:
I20260808 10:12:00.684147 24105 leader_election.cc:288] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:12:00.684173 24110 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:12:00.684294 24110 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [term 1 LEADER]: Becoming Leader. State: Replica: a0f9626d7e9242b5b6562f9fe3649774, State: Running, Role: LEADER
I20260808 10:12:00.684433 24110 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f9626d7e9242b5b6562f9fe3649774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } }
I20260808 10:12:00.684419 24105 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:00.684933 24111 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0f9626d7e9242b5b6562f9fe3649774" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f9626d7e9242b5b6562f9fe3649774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } } }
I20260808 10:12:00.685041 24111 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:00.684937 24112 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0f9626d7e9242b5b6562f9fe3649774. Latest consensus state: current_term: 1 leader_uuid: "a0f9626d7e9242b5b6562f9fe3649774" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0f9626d7e9242b5b6562f9fe3649774" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } } }
I20260808 10:12:00.685266 24112 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:00.685436 24115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:12:00.685953 24115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:12:00.687614 24115 catalog_manager.cc:1347] Generated new cluster ID: c379d36175c048b88d6f22c665d62e43
I20260808 10:12:00.687677 24115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:12:00.688140 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:00.681930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:42403 (local address 127.0.0.1:42111)
0808 10:12:00.682146 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:00.682155 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:00.682191 (+ 36us) server_negotiation.cc:407] Connection header received
0808 10:12:00.682548 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:00.682556 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:00.682621 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:00.682783 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:00.683085 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:00.684591 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:00.685819 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:00.686416 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:00.686543 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:00.687345 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:00.687348 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:00.687349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:12:00.687558 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:00.687600 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:12:00.687616 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:00.687704 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:00.687906 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 10:12:00.687932 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":106,"threads_started":1}
I20260808 10:12:00.689146 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:12:00.694303 24109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:00.690798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:33645 (local address 127.0.0.1:42111)
0808 10:12:00.690830 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:00.690832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:00.690878 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:12:00.690948 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:00.690949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:00.690959 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:00.691032 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:00.691202 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:00.691927 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:00.692904 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:00.693230 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:00.693266 (+ 36us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:00.694046 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:00.694047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:00.694048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:12:00.694052 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:00.694065 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:12:00.694069 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:00.694097 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:00.694213 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 10:12:00.694226 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 10:12:00.696259 24115 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:12:00.696744 24115 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:12:00.703101 24115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0f9626d7e9242b5b6562f9fe3649774: Generated new TSK 0
I20260808 10:12:00.703287 24115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:12:00.709367 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 24036
W20260808 10:12:00.715808 24020 connection.cc:535] client connection to 127.0.0.1:42111 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42111): Connection reset by peer (error 104)
2026-08-08T10:12:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:12:00 dist-test-slave-w206 krb5kdc[24138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:12:00 dist-test-slave-w206 krb5kdc[24138](info): set up 2 sockets
Aug 08 10:12:00 dist-test-slave-w206 krb5kdc[24138](info): commencing operation
krb5kdc: starting...
W20260808 10:12:01.776487 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:12:01 dist-test-slave-w206 krb5kdc[24138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:12:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:12:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:12:01.915143 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34649
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:12:02.004062 24152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:02.004352 24152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:02.004393 24152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:02.005632 24152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:02.005723 24152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:02.005749 24152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:02.005772 24152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:02.005801 24152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:02.007507 24152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:02.007833 24152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:02.008104 24152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:12:02.013603 24158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:02.013810 24157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:02.013947 24152 server_base.cc:1034] running on GCE node
W20260808 10:12:02.013603 24160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:02.014253 24152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:02.014607 24152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:02.015827 24152 hybrid_clock.cc:648] HybridClock initialized: now 1786183922015803 us; error 42 us; skew 500 ppm
Aug 08 10:12:02 dist-test-slave-w206 krb5kdc[24138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183922, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:02.022617 24152 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:12:02.023198 24152 webserver.cc:460] Webserver started at http://127.0.0.1:41697/ using document root <none> and password file <none>
I20260808 10:12:02.023445 24152 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:02.023491 24152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:02.023591 24152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:02.024708 24152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "61d5577cb64043ab9d45426a1282aa9a"
format_stamp: "Formatted at 2026-08-08 10:12:02 on dist-test-slave-w206"
server_key: "5f989b8ee599274b2c0bbbe6b09deb46"
server_key_iv: "38c20162ff23a75ce6fa6a4af5dd3d30"
server_key_version: "encryptionkey@0"
I20260808 10:12:02.025035 24152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "61d5577cb64043ab9d45426a1282aa9a"
format_stamp: "Formatted at 2026-08-08 10:12:02 on dist-test-slave-w206"
server_key: "5f989b8ee599274b2c0bbbe6b09deb46"
server_key_iv: "38c20162ff23a75ce6fa6a4af5dd3d30"
server_key_version: "encryptionkey@0"
I20260808 10:12:02.026469 24152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:12:02.027410 24167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:02.027647 24152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:12:02.027751 24152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "61d5577cb64043ab9d45426a1282aa9a"
format_stamp: "Formatted at 2026-08-08 10:12:02 on dist-test-slave-w206"
server_key: "5f989b8ee599274b2c0bbbe6b09deb46"
server_key_iv: "38c20162ff23a75ce6fa6a4af5dd3d30"
server_key_version: "encryptionkey@0"
I20260808 10:12:02.027828 24152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:02.031385 24152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:02.034554 24152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:02.034806 24152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:02.039578 24152 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34649
I20260808 10:12:02.039650 24222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34649 every 8 connection(s)
I20260808 10:12:02.040024 24152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:12:02.041139 24223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:02.042043 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 24152
I20260808 10:12:02.042243 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:12:02.042496 2170 external_mini_cluster.cc:1428] Setting key 75b2b1a4cfb30d61062191cc9ab7c16c
I20260808 10:12:02.044909 24223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a: Bootstrap starting.
I20260808 10:12:02.045588 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:02.043126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:55437 (local address 127.0.0.1:34649)
0808 10:12:02.043352 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:02.043362 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:02.043396 (+ 34us) server_negotiation.cc:407] Connection header received
0808 10:12:02.043982 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:02.043992 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:02.044031 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:02.044199 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:02.044947 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:02.044951 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:02.044951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:12:02.044956 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:02.045006 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:12:02.045021 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:02.045094 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:02.045324 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 10:12:02.045379 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":105,"threads_started":1}
I20260808 10:12:02.045776 24223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:02.046058 24223 log.cc:824] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:02.046902 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:12:02.047088 24223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a: No bootstrap required, opened a new log
I20260808 10:12:02.049755 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:02.048409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:32895 (local address 127.0.0.1:34649)
0808 10:12:02.048454 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:02.048456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:02.048581 (+ 125us) server_negotiation.cc:407] Connection header received
0808 10:12:02.048696 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:02.048696 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:02.048701 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:02.048750 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:02.049529 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:02.049530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:02.049530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:12:02.049534 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:02.049549 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:12:02.049553 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:02.049614 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:02.049687 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 10:12:02.049699 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 10:12:02.049786 24223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d5577cb64043ab9d45426a1282aa9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34649 } }
I20260808 10:12:02.050031 24223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:12:02.050081 24223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61d5577cb64043ab9d45426a1282aa9a, State: Initialized, Role: FOLLOWER
I20260808 10:12:02.050215 24223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d5577cb64043ab9d45426a1282aa9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34649 } }
I20260808 10:12:02.050325 24223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:12:02.050374 24223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:12:02.050426 24223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:02.051157 24223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d5577cb64043ab9d45426a1282aa9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34649 } }
I20260808 10:12:02.051288 24223 leader_election.cc:302] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61d5577cb64043ab9d45426a1282aa9a; no voters:
I20260808 10:12:02.051522 24223 leader_election.cc:288] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:12:02.051556 24233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:12:02.051714 24233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [term 1 LEADER]: Becoming Leader. State: Replica: 61d5577cb64043ab9d45426a1282aa9a, State: Running, Role: LEADER
I20260808 10:12:02.051803 24233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d5577cb64043ab9d45426a1282aa9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34649 } }
I20260808 10:12:02.051838 24223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:02.052175 24233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61d5577cb64043ab9d45426a1282aa9a. Latest consensus state: current_term: 1 leader_uuid: "61d5577cb64043ab9d45426a1282aa9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d5577cb64043ab9d45426a1282aa9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34649 } } }
I20260808 10:12:02.052201 24235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61d5577cb64043ab9d45426a1282aa9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61d5577cb64043ab9d45426a1282aa9a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34649 } } }
I20260808 10:12:02.052254 24233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:02.052482 24235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:02.052507 24237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:12:02.053033 24237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:12:02.054874 24237 catalog_manager.cc:1347] Generated new cluster ID: dfb0ca96261049a28c6e5cfc337ec33b
I20260808 10:12:02.054958 24237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:12:02.094813 24237 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:12:02.095069 24237 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:12:02.099164 24237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61d5577cb64043ab9d45426a1282aa9a: Generated new TSK 0
I20260808 10:12:02.099377 24237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:12:02.110061 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 24152
2026-08-08T10:12:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1400 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:12:02 dist-test-slave-w206 krb5kdc[24255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:12:02 dist-test-slave-w206 krb5kdc[24255](info): set up 2 sockets
Aug 08 10:12:02 dist-test-slave-w206 krb5kdc[24255](info): commencing operation
krb5kdc: starting...
W20260808 10:12:03.196296 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:12:03 dist-test-slave-w206 krb5kdc[24255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:12:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:12:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:12:03.334748 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37155
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:12:03.420958 24269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:03.421218 24269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:03.421263 24269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:03.422492 24269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:03.422571 24269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:03.422595 24269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:03.422626 24269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:03.422658 24269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:03.424286 24269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37155
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:03.424587 24269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:03.424832 24269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:12:03.429993 24275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:03.430001 24276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:03.430013 24278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:03.430393 24269 server_base.cc:1034] running on GCE node
I20260808 10:12:03.430694 24269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:03.431160 24269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:03.432384 24269 hybrid_clock.cc:648] HybridClock initialized: now 1786183923432381 us; error 43 us; skew 500 ppm
Aug 08 10:12:03 dist-test-slave-w206 krb5kdc[24255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183923, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:03.435287 24269 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:12:03.435791 24269 webserver.cc:460] Webserver started at http://127.0.0.1:41471/ using document root <none> and password file <none>
I20260808 10:12:03.436074 24269 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:03.436153 24269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:03.436347 24269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:03.437574 24269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "2e54e1b01a33463dbb6677d16c9cdbf4"
format_stamp: "Formatted at 2026-08-08 10:12:03 on dist-test-slave-w206"
server_key: "fb30ee6437cd67385f8ddb1973a0b85d"
server_key_iv: "17b4d2858edb880db5c0ec5f213ea865"
server_key_version: "encryptionkey@0"
I20260808 10:12:03.437983 24269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "2e54e1b01a33463dbb6677d16c9cdbf4"
format_stamp: "Formatted at 2026-08-08 10:12:03 on dist-test-slave-w206"
server_key: "fb30ee6437cd67385f8ddb1973a0b85d"
server_key_iv: "17b4d2858edb880db5c0ec5f213ea865"
server_key_version: "encryptionkey@0"
I20260808 10:12:03.439601 24269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:12:03.440510 24285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:03.440753 24269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:12:03.440865 24269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "2e54e1b01a33463dbb6677d16c9cdbf4"
format_stamp: "Formatted at 2026-08-08 10:12:03 on dist-test-slave-w206"
server_key: "fb30ee6437cd67385f8ddb1973a0b85d"
server_key_iv: "17b4d2858edb880db5c0ec5f213ea865"
server_key_version: "encryptionkey@0"
I20260808 10:12:03.440951 24269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:03.456211 24269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:03.458706 24269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:03.458881 24269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:03.463084 24269 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37155
I20260808 10:12:03.463145 24340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37155 every 8 connection(s)
I20260808 10:12:03.463500 24269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:12:03.464502 24341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:03.467519 24341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4: Bootstrap starting.
I20260808 10:12:03.468104 24341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:03.468392 24341 log.cc:824] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:03.469249 24341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4: No bootstrap required, opened a new log
I20260808 10:12:03.471061 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 24269
I20260808 10:12:03.471231 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:12:03.471254 24341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e54e1b01a33463dbb6677d16c9cdbf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37155 } }
I20260808 10:12:03.471434 2170 external_mini_cluster.cc:1428] Setting key d11ac44e1de74d1275a7f133598a9277
I20260808 10:12:03.471448 24341 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:12:03.471479 24341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e54e1b01a33463dbb6677d16c9cdbf4, State: Initialized, Role: FOLLOWER
I20260808 10:12:03.471662 24341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e54e1b01a33463dbb6677d16c9cdbf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37155 } }
I20260808 10:12:03.471762 24341 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:12:03.471803 24341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:12:03.471922 24341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:03.472708 24341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e54e1b01a33463dbb6677d16c9cdbf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37155 } }
I20260808 10:12:03.472892 24341 leader_election.cc:302] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e54e1b01a33463dbb6677d16c9cdbf4; no voters:
I20260808 10:12:03.473129 24341 leader_election.cc:288] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:12:03.473146 24346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:12:03.473306 24346 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [term 1 LEADER]: Becoming Leader. State: Replica: 2e54e1b01a33463dbb6677d16c9cdbf4, State: Running, Role: LEADER
I20260808 10:12:03.473445 24341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:03.473429 24346 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e54e1b01a33463dbb6677d16c9cdbf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37155 } }
I20260808 10:12:03.473923 24348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e54e1b01a33463dbb6677d16c9cdbf4. Latest consensus state: current_term: 1 leader_uuid: "2e54e1b01a33463dbb6677d16c9cdbf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e54e1b01a33463dbb6677d16c9cdbf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37155 } } }
I20260808 10:12:03.474071 24348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:03.474323 24347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e54e1b01a33463dbb6677d16c9cdbf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e54e1b01a33463dbb6677d16c9cdbf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37155 } } }
I20260808 10:12:03.474474 24347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:03.474647 24353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:12:03.475373 24353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:12:03.477190 24353 catalog_manager.cc:1347] Generated new cluster ID: d11038950b4044e588b33d44610f302b
I20260808 10:12:03.477257 24353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:12:03 dist-test-slave-w206 krb5kdc[24255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 10:12:03.485320 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:03.471969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:38067 (local address 127.0.0.1:37155)
0808 10:12:03.472190 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:03.472200 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:03.472246 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:12:03.472608 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:03.472614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:03.472662 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:03.472804 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:03.473933 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:03.475678 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:03.476840 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:03.477478 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:03.477602 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:03.480727 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:03.480732 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:03.480732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:03.480752 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:03.483019 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:03.483502 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:03.483505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:03.483506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:03.483680 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:03.484165 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:03.484166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:03.484166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:03.484502 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:03.484666 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:03.484826 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 10:12:03.485052 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":119,"threads_started":1}
I20260808 10:12:03.486419 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:12:03.492733 24353 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:12:03.493507 24353 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:12:03.495848 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:03.488062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:42037 (local address 127.0.0.1:37155)
0808 10:12:03.488100 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:03.488102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:03.488135 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:12:03.488212 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:03.488212 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:03.488222 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:03.488293 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:03.488961 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:03.489742 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:03.490815 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:03.491164 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:03.491243 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:03.492736 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:03.492737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:03.492738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:03.492744 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:03.494479 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:03.494910 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:03.494911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:03.494912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:03.494964 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:03.495248 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:03.495249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:03.495250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:03.495369 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:03.495455 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:03.495691 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 10:12:03.495756 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:12:03.507012 24353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e54e1b01a33463dbb6677d16c9cdbf4: Generated new TSK 0
I20260808 10:12:03.507264 24353 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:12:03.511153 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 24269
W20260808 10:12:03.519815 24253 connection.cc:535] client connection to 127.0.0.1:37155 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37155): Connection reset by peer (error 104)
2026-08-08T10:12:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:12:03 dist-test-slave-w206 krb5kdc[24372](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:12:03 dist-test-slave-w206 krb5kdc[24372](info): set up 2 sockets
Aug 08 10:12:03 dist-test-slave-w206 krb5kdc[24372](info): commencing operation
krb5kdc: starting...
W20260808 10:12:04.610958 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:12:04 dist-test-slave-w206 krb5kdc[24372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:12:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:12:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:12:04.746886 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45129
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:12:04.833364 24386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:04.833601 24386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:04.833627 24386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:04.834844 24386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:04.834888 24386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:04.834903 24386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:04.834916 24386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:04.834928 24386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:04.836434 24386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45129
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:04.836648 24386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:04.836843 24386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:12:04.842278 24392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:04.842213 24394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:04.842232 24386 server_base.cc:1034] running on GCE node
W20260808 10:12:04.842206 24391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:04.842707 24386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:04.843089 24386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:04.844342 24386 hybrid_clock.cc:648] HybridClock initialized: now 1786183924844301 us; error 70 us; skew 500 ppm
Aug 08 10:12:04 dist-test-slave-w206 krb5kdc[24372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183924, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:04.847131 24386 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:12:04.847563 24386 webserver.cc:460] Webserver started at http://127.0.0.1:37489/ using document root <none> and password file <none>
I20260808 10:12:04.847762 24386 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:04.847805 24386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:04.847903 24386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:04.848992 24386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "cce141bc587d4b5890f84c871c32c909"
format_stamp: "Formatted at 2026-08-08 10:12:04 on dist-test-slave-w206"
server_key: "080c358bd341d9a4da7613cb72ebe7c7"
server_key_iv: "42791fc44f70434f1601038ad95918e2"
server_key_version: "encryptionkey@0"
I20260808 10:12:04.849313 24386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "cce141bc587d4b5890f84c871c32c909"
format_stamp: "Formatted at 2026-08-08 10:12:04 on dist-test-slave-w206"
server_key: "080c358bd341d9a4da7613cb72ebe7c7"
server_key_iv: "42791fc44f70434f1601038ad95918e2"
server_key_version: "encryptionkey@0"
I20260808 10:12:04.850759 24386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:12:04.851833 24402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:04.852072 24386 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:12:04.852190 24386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "cce141bc587d4b5890f84c871c32c909"
format_stamp: "Formatted at 2026-08-08 10:12:04 on dist-test-slave-w206"
server_key: "080c358bd341d9a4da7613cb72ebe7c7"
server_key_iv: "42791fc44f70434f1601038ad95918e2"
server_key_version: "encryptionkey@0"
I20260808 10:12:04.852299 24386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:04.879411 24386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:04.882102 24386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:04.882328 24386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:04.887542 24386 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45129
I20260808 10:12:04.887611 24457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45129 every 8 connection(s)
I20260808 10:12:04.888024 24386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:12:04.888993 24459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:04.891486 24459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909: Bootstrap starting.
I20260808 10:12:04.892097 24459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:04.892407 24459 log.cc:824] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:04.893174 24459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909: No bootstrap required, opened a new log
I20260808 10:12:04.893750 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 24386
I20260808 10:12:04.893889 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:12:04.894104 2170 external_mini_cluster.cc:1428] Setting key 22261fa1f96bf38ef05c39e158c1cded
I20260808 10:12:04.895582 24459 raft_consensus.cc:348] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce141bc587d4b5890f84c871c32c909" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45129 } }
I20260808 10:12:04.895794 24459 raft_consensus.cc:374] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:12:04.895869 24459 raft_consensus.cc:729] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cce141bc587d4b5890f84c871c32c909, State: Initialized, Role: FOLLOWER
I20260808 10:12:04.896067 24459 consensus_queue.cc:260] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce141bc587d4b5890f84c871c32c909" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45129 } }
I20260808 10:12:04.896237 24459 raft_consensus.cc:388] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:12:04.896341 24459 raft_consensus.cc:482] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:12:04.896401 24459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:04.897018 24459 raft_consensus.cc:504] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce141bc587d4b5890f84c871c32c909" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45129 } }
I20260808 10:12:04.897142 24459 leader_election.cc:302] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cce141bc587d4b5890f84c871c32c909; no voters:
I20260808 10:12:04.897344 24459 leader_election.cc:288] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:12:04.897377 24465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:12:04.897487 24465 raft_consensus.cc:686] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [term 1 LEADER]: Becoming Leader. State: Replica: cce141bc587d4b5890f84c871c32c909, State: Running, Role: LEADER
I20260808 10:12:04.897620 24459 sys_catalog.cc:564] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:04.897636 24465 consensus_queue.cc:237] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce141bc587d4b5890f84c871c32c909" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45129 } }
I20260808 10:12:04.898144 24465 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cce141bc587d4b5890f84c871c32c909. Latest consensus state: current_term: 1 leader_uuid: "cce141bc587d4b5890f84c871c32c909" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce141bc587d4b5890f84c871c32c909" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45129 } } }
I20260808 10:12:04.898321 24465 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:04.898571 24475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:12:04.899163 24475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:12:04.899219 24466 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cce141bc587d4b5890f84c871c32c909" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cce141bc587d4b5890f84c871c32c909" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45129 } } }
I20260808 10:12:04.899528 24466 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:04.901248 24475 catalog_manager.cc:1347] Generated new cluster ID: 25e5136f1dc047518098adcbae072157
I20260808 10:12:04.901330 24475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:12:04.901515 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:04.894763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:56759 (local address 127.0.0.1:45129)
0808 10:12:04.895008 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:04.895018 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:04.895055 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:12:04.895651 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:04.895661 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:04.895739 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:04.895890 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:04.896203 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:04.897811 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:04.899006 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:04.899683 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:04.899822 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:04.900637 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:04.900641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:04.900641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:12:04.900898 (+ 257us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:04.900941 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:12:04.900956 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:04.901039 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:04.901230 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 10:12:04.901259 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":134,"threads_started":1}
I20260808 10:12:04.902571 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:12:04.910873 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:04.904131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:44737 (local address 127.0.0.1:45129)
0808 10:12:04.904168 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:04.904170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:04.907205 (+ 3035us) server_negotiation.cc:407] Connection header received
0808 10:12:04.907259 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:04.907260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:04.907273 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:04.907355 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:04.907516 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:04.908316 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:04.909333 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:04.909711 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:04.909790 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:04.910593 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:04.910594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:04.910595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:12:04.910600 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:04.910619 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:12:04.910623 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:04.910690 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:04.910762 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 10:12:04.910777 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 10:12:04.919056 24475 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:12:04.919632 24475 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:12:04.924256 24475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cce141bc587d4b5890f84c871c32c909: Generated new TSK 0
I20260808 10:12:04.924480 24475 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:12:04.929226 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 24386
W20260808 10:12:04.935796 24370 connection.cc:535] client connection to 127.0.0.1:45129 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45129): Connection reset by peer (error 104)
2026-08-08T10:12:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1415 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:12:04 dist-test-slave-w206 krb5kdc[24494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:12:04 dist-test-slave-w206 krb5kdc[24494](info): set up 2 sockets
Aug 08 10:12:04 dist-test-slave-w206 krb5kdc[24494](info): commencing operation
krb5kdc: starting...
W20260808 10:12:06.007617 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:12:06 dist-test-slave-w206 krb5kdc[24494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:12:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:12:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 10:12:06.142539 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33937
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:12:06.229118 24507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:06.229403 24507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:06.229449 24507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:06.230679 24507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:06.230757 24507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:06.230782 24507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:06.230803 24507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:06.230834 24507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:06.233598 24507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33937
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:06.233933 24507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:06.234256 24507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:12:06.240849 24513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:06.240849 24515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:06.240938 24512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:06.241289 24507 server_base.cc:1034] running on GCE node
I20260808 10:12:06.241561 24507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:06.241842 24507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:06.243052 24507 hybrid_clock.cc:648] HybridClock initialized: now 1786183926243032 us; error 36 us; skew 500 ppm
Aug 08 10:12:06 dist-test-slave-w206 krb5kdc[24494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183926, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:06.245745 24507 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 10:12:06.246282 24507 webserver.cc:460] Webserver started at http://127.0.0.1:45267/ using document root <none> and password file <none>
I20260808 10:12:06.246526 24507 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:06.246573 24507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:06.246713 24507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:06.247920 24507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "3356a626fe5a491ca5f77e9862c8e899"
format_stamp: "Formatted at 2026-08-08 10:12:06 on dist-test-slave-w206"
server_key: "690f51ba78819160ce18a1401b664ea7"
server_key_iv: "941a4cf919193a0aa68b43a09bf2ee8d"
server_key_version: "encryptionkey@0"
I20260808 10:12:06.248296 24507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "3356a626fe5a491ca5f77e9862c8e899"
format_stamp: "Formatted at 2026-08-08 10:12:06 on dist-test-slave-w206"
server_key: "690f51ba78819160ce18a1401b664ea7"
server_key_iv: "941a4cf919193a0aa68b43a09bf2ee8d"
server_key_version: "encryptionkey@0"
I20260808 10:12:06.249782 24507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 10:12:06.250687 24522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:06.250905 24507 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:12:06.251040 24507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "3356a626fe5a491ca5f77e9862c8e899"
format_stamp: "Formatted at 2026-08-08 10:12:06 on dist-test-slave-w206"
server_key: "690f51ba78819160ce18a1401b664ea7"
server_key_iv: "941a4cf919193a0aa68b43a09bf2ee8d"
server_key_version: "encryptionkey@0"
I20260808 10:12:06.251122 24507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:06.254678 24507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:06.256956 24507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:06.257121 24507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:06.261462 24507 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33937
I20260808 10:12:06.261536 24576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33937 every 8 connection(s)
I20260808 10:12:06.261885 24507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:12:06.262909 24577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:06.265293 24577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899: Bootstrap starting.
I20260808 10:12:06.265933 24577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:06.266255 24577 log.cc:824] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:06.267148 24577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899: No bootstrap required, opened a new log
I20260808 10:12:06.269045 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 24507
I20260808 10:12:06.269196 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:12:06.269438 2170 external_mini_cluster.cc:1428] Setting key 43257b9052abbb4ae4328b6a314c648d
I20260808 10:12:06.269280 24577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3356a626fe5a491ca5f77e9862c8e899" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33937 } }
I20260808 10:12:06.269481 24577 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:12:06.269526 24577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3356a626fe5a491ca5f77e9862c8e899, State: Initialized, Role: FOLLOWER
I20260808 10:12:06.269738 24577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3356a626fe5a491ca5f77e9862c8e899" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33937 } }
I20260808 10:12:06.269872 24577 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:12:06.269913 24577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:12:06.269958 24577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:06.270709 24577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3356a626fe5a491ca5f77e9862c8e899" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33937 } }
I20260808 10:12:06.270849 24577 leader_election.cc:302] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3356a626fe5a491ca5f77e9862c8e899; no voters:
I20260808 10:12:06.271068 24577 leader_election.cc:288] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:12:06.271173 24582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:12:06.271355 24582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [term 1 LEADER]: Becoming Leader. State: Replica: 3356a626fe5a491ca5f77e9862c8e899, State: Running, Role: LEADER
I20260808 10:12:06.271384 24577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:06.271499 24582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3356a626fe5a491ca5f77e9862c8e899" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33937 } }
I20260808 10:12:06.271870 24582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3356a626fe5a491ca5f77e9862c8e899. Latest consensus state: current_term: 1 leader_uuid: "3356a626fe5a491ca5f77e9862c8e899" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3356a626fe5a491ca5f77e9862c8e899" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33937 } } }
I20260808 10:12:06.271962 24582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:06.271886 24583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3356a626fe5a491ca5f77e9862c8e899" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3356a626fe5a491ca5f77e9862c8e899" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33937 } } }
I20260808 10:12:06.272158 24583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:06.272202 24588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:12:06.272423 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:06.270002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:46447 (local address 127.0.0.1:33937)
0808 10:12:06.270210 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:06.270220 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:06.270285 (+ 65us) server_negotiation.cc:407] Connection header received
0808 10:12:06.270651 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:06.270657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:06.270683 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:06.270837 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:06.271863 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:06.271867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:06.271867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:12:06.271872 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:06.271922 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:12:06.271937 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:06.272013 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:06.272147 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 10:12:06.272197 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":113,"threads_started":1}
I20260808 10:12:06.272759 24588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:12:06.273869 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:12:06.275046 24588 catalog_manager.cc:1347] Generated new cluster ID: 7137c5cfd8ab4912b914d3aa93d72fdb
I20260808 10:12:06.275094 24588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:12:06.280267 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:06.275619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.133:36233 (local address 127.0.0.1:33937)
0808 10:12:06.279068 (+ 3449us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:06.279070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:06.279081 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:12:06.279108 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:06.279110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:06.279115 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:06.279181 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:06.279998 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:06.280000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:06.280000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:12:06.280005 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:06.280021 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:12:06.280026 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:06.280081 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:06.280159 (+ 78us) server_negotiation.cc:299] Negotiation successful
0808 10:12:06.280173 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3420}
I20260808 10:12:06.293818 24588 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:12:06.294044 24588 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:12:06.302680 24588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3356a626fe5a491ca5f77e9862c8e899: Generated new TSK 0
I20260808 10:12:06.302938 24588 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:12:06.313365 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 24507
2026-08-08T10:12:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1384 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:12:06 dist-test-slave-w206 krb5kdc[24608](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:12:06 dist-test-slave-w206 krb5kdc[24608](info): set up 2 sockets
Aug 08 10:12:06 dist-test-slave-w206 krb5kdc[24608](info): commencing operation
krb5kdc: starting...
W20260808 10:12:07.411564 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:12:07 dist-test-slave-w206 krb5kdc[24608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:12:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:12:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:12:07.542133 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:45895
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:42235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:45895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:12:07.626471 24624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:07.626752 24624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:07.626796 24624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:07.628000 24624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:07.628089 24624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:07.628119 24624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:07.628163 24624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:07.628193 24624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:07.629747 24624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:42235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:45895
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:45895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:07.630048 24624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:07.630308 24624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:12:07.635934 24630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:07.635926 24633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:07.635993 24631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:07.636358 24624 server_base.cc:1034] running on GCE node
I20260808 10:12:07.636610 24624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:07.636914 24624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:07.638067 24624 hybrid_clock.cc:648] HybridClock initialized: now 1786183927638041 us; error 35 us; skew 500 ppm
Aug 08 10:12:07 dist-test-slave-w206 krb5kdc[24608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183927, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:07.644960 24624 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:12:07.645468 24624 webserver.cc:460] Webserver started at http://127.2.30.190:38451/ using document root <none> and password file <none>
I20260808 10:12:07.645717 24624 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:07.645785 24624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:07.645934 24624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:07.647102 24624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "30063a8dcb7347679c4449255a140b5c"
format_stamp: "Formatted at 2026-08-08 10:12:07 on dist-test-slave-w206"
server_key: "a30f25c8f6440d05647f0f6e3b6f2064"
server_key_iv: "f2ccfd90c8894c7ec50ae97be0dcb4e8"
server_key_version: "encryptionkey@0"
I20260808 10:12:07.647490 24624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "30063a8dcb7347679c4449255a140b5c"
format_stamp: "Formatted at 2026-08-08 10:12:07 on dist-test-slave-w206"
server_key: "a30f25c8f6440d05647f0f6e3b6f2064"
server_key_iv: "f2ccfd90c8894c7ec50ae97be0dcb4e8"
server_key_version: "encryptionkey@0"
I20260808 10:12:07.648871 24624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:12:07.649660 24640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:07.649955 24624 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:12:07.650053 24624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "30063a8dcb7347679c4449255a140b5c"
format_stamp: "Formatted at 2026-08-08 10:12:07 on dist-test-slave-w206"
server_key: "a30f25c8f6440d05647f0f6e3b6f2064"
server_key_iv: "f2ccfd90c8894c7ec50ae97be0dcb4e8"
server_key_version: "encryptionkey@0"
I20260808 10:12:07.650127 24624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:07.664851 24624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:07.667052 24624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:07.667243 24624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:07.671644 24624 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:45895
I20260808 10:12:07.671693 24696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:45895 every 8 connection(s)
I20260808 10:12:07.672037 24624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:12:07.672868 24697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:07.675285 24697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c: Bootstrap starting.
I20260808 10:12:07.675876 24697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:07.676190 24697 log.cc:824] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:07.676916 24697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c: No bootstrap required, opened a new log
I20260808 10:12:07.678328 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 24624
I20260808 10:12:07.678476 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:12:07.678684 2170 external_mini_cluster.cc:1428] Setting key 89250fe2dc6e272f4e55254411450a4e
I20260808 10:12:07.678952 24697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30063a8dcb7347679c4449255a140b5c" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45895 } }
I20260808 10:12:07.679173 24697 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:12:07.679239 24697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30063a8dcb7347679c4449255a140b5c, State: Initialized, Role: FOLLOWER
I20260808 10:12:07.679442 24697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30063a8dcb7347679c4449255a140b5c" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45895 } }
I20260808 10:12:07.679548 24697 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:12:07.679575 24697 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:12:07.679646 24697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:07.680485 24697 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30063a8dcb7347679c4449255a140b5c" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45895 } }
I20260808 10:12:07.680606 24697 leader_election.cc:302] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30063a8dcb7347679c4449255a140b5c; no voters:
I20260808 10:12:07.680789 24697 leader_election.cc:288] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:12:07.680838 24702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:12:07.681021 24702 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [term 1 LEADER]: Becoming Leader. State: Replica: 30063a8dcb7347679c4449255a140b5c, State: Running, Role: LEADER
I20260808 10:12:07.681082 24697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:07.681133 24702 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30063a8dcb7347679c4449255a140b5c" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45895 } }
I20260808 10:12:07.681564 24702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30063a8dcb7347679c4449255a140b5c. Latest consensus state: current_term: 1 leader_uuid: "30063a8dcb7347679c4449255a140b5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30063a8dcb7347679c4449255a140b5c" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45895 } } }
I20260808 10:12:07.681593 24703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30063a8dcb7347679c4449255a140b5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30063a8dcb7347679c4449255a140b5c" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 45895 } } }
I20260808 10:12:07.681704 24702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:07.681936 24703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:07.681977 24711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:12:07.682559 24711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:12:07.684484 24711 catalog_manager.cc:1347] Generated new cluster ID: 19a7a987808a4c3a927af542bb3821cf
I20260808 10:12:07.684535 24711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 10:12:07 dist-test-slave-w206 krb5kdc[24608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:12:07.693817 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:07.679270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59997 (local address 127.2.30.190:45895)
0808 10:12:07.679528 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:07.679540 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:07.679577 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:12:07.679713 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:07.679721 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:07.679788 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:07.679931 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:07.680789 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:07.682270 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:07.683374 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:07.683961 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:07.684074 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:07.689630 (+ 5556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:07.689636 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:07.689637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:07.689653 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:07.691860 (+ 2207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:07.692411 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:07.692414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:07.692415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:07.692521 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:07.692763 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:07.692763 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:07.692764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:07.693099 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:07.693273 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:07.693423 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 10:12:07.693523 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":179,"threads_started":1}
I20260808 10:12:07.694938 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:12:07.703900 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:07.696505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52765 (local address 127.2.30.190:45895)
0808 10:12:07.696548 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:07.696550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:07.696640 (+ 90us) server_negotiation.cc:407] Connection header received
0808 10:12:07.696698 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:07.696699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:07.696709 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:07.696764 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:07.697421 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:07.698163 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:07.699221 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:07.699537 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:07.699599 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:07.701101 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:07.701102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:07.701102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:07.701109 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:07.702651 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:07.703078 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:07.703079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:07.703079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:07.703133 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:07.703376 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:07.703377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:07.703377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:07.703496 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:07.703585 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:07.703742 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 10:12:07.703802 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:12:07.711951 24711 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:12:07.712716 24711 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:12:07.717013 24711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30063a8dcb7347679c4449255a140b5c: Generated new TSK 0
I20260808 10:12:07.717206 24711 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:12:07.720007 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 24624
W20260808 10:12:07.726136 24606 connection.cc:535] client connection to 127.2.30.190:45895 recv error: Network error: failed to read from TLS socket (remote: 127.2.30.190:45895): Connection reset by peer (error 104)
2026-08-08T10:12:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1406 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:12:07 dist-test-slave-w206 krb5kdc[24728](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:12:07 dist-test-slave-w206 krb5kdc[24728](info): set up 2 sockets
Aug 08 10:12:07 dist-test-slave-w206 krb5kdc[24728](info): commencing operation
krb5kdc: starting...
W20260808 10:12:08.804337 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:12:08 dist-test-slave-w206 krb5kdc[24728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:12:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:12:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:12:08.939396 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:36377
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:43361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:36377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:12:09.024959 24742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:09.025236 24742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:09.025279 24742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:09.026450 24742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:09.026549 24742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:09.026580 24742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:09.026614 24742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:09.026641 24742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:09.028296 24742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:43361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:36377
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:36377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:09.028605 24742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:09.028859 24742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:12:09.034096 24748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:09.034090 24749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:09.034269 24742 server_base.cc:1034] running on GCE node
W20260808 10:12:09.034353 24751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:09.034585 24742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:09.034904 24742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:09.036110 24742 hybrid_clock.cc:648] HybridClock initialized: now 1786183929036062 us; error 63 us; skew 500 ppm
Aug 08 10:12:09 dist-test-slave-w206 krb5kdc[24728](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183929, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:09.038947 24742 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:12:09.039462 24742 webserver.cc:460] Webserver started at http://127.2.30.190:44879/ using document root <none> and password file <none>
I20260808 10:12:09.039685 24742 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:09.039731 24742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:09.039824 24742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:09.040971 24742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "1d17b09cb2804249813f257bed6da293"
format_stamp: "Formatted at 2026-08-08 10:12:09 on dist-test-slave-w206"
server_key: "2a82c7742e84fb3324718d0718c45f4c"
server_key_iv: "370fd6670ec773daa864b9cb2bf650e7"
server_key_version: "encryptionkey@0"
I20260808 10:12:09.041313 24742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "1d17b09cb2804249813f257bed6da293"
format_stamp: "Formatted at 2026-08-08 10:12:09 on dist-test-slave-w206"
server_key: "2a82c7742e84fb3324718d0718c45f4c"
server_key_iv: "370fd6670ec773daa864b9cb2bf650e7"
server_key_version: "encryptionkey@0"
I20260808 10:12:09.042768 24742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:12:09.043648 24758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:09.043818 24742 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 10:12:09.043911 24742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "1d17b09cb2804249813f257bed6da293"
format_stamp: "Formatted at 2026-08-08 10:12:09 on dist-test-slave-w206"
server_key: "2a82c7742e84fb3324718d0718c45f4c"
server_key_iv: "370fd6670ec773daa864b9cb2bf650e7"
server_key_version: "encryptionkey@0"
I20260808 10:12:09.043977 24742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:09.053879 24742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:09.056169 24742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:09.056367 24742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:09.060497 24742 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:36377
I20260808 10:12:09.060577 24814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:36377 every 8 connection(s)
I20260808 10:12:09.061048 24742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:12:09.061916 24815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:09.065158 24815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293: Bootstrap starting.
I20260808 10:12:09.065629 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 24742
I20260808 10:12:09.065775 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:12:09.065872 24815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:09.066045 2170 external_mini_cluster.cc:1428] Setting key 00a8ed5e04aed1190e5ba72d32ee7566
I20260808 10:12:09.066232 24815 log.cc:824] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:09.067113 24815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293: No bootstrap required, opened a new log
I20260808 10:12:09.069777 24815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d17b09cb2804249813f257bed6da293" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36377 } }
I20260808 10:12:09.069909 24815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:12:09.069931 24815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d17b09cb2804249813f257bed6da293, State: Initialized, Role: FOLLOWER
I20260808 10:12:09.070062 24815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d17b09cb2804249813f257bed6da293" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36377 } }
I20260808 10:12:09.070138 24815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:12:09.070179 24815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:12:09.070227 24815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:09.070963 24815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d17b09cb2804249813f257bed6da293" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36377 } }
I20260808 10:12:09.071108 24815 leader_election.cc:302] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d17b09cb2804249813f257bed6da293; no voters:
I20260808 10:12:09.071286 24815 leader_election.cc:288] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:12:09.071323 24821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:12:09.071547 24821 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [term 1 LEADER]: Becoming Leader. State: Replica: 1d17b09cb2804249813f257bed6da293, State: Running, Role: LEADER
I20260808 10:12:09.071583 24815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:09.071661 24821 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d17b09cb2804249813f257bed6da293" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36377 } }
I20260808 10:12:09.072021 24822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d17b09cb2804249813f257bed6da293" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d17b09cb2804249813f257bed6da293" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36377 } } }
I20260808 10:12:09.072124 24822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:09.072436 24828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:12:09.072793 24823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d17b09cb2804249813f257bed6da293. Latest consensus state: current_term: 1 leader_uuid: "1d17b09cb2804249813f257bed6da293" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d17b09cb2804249813f257bed6da293" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 36377 } } }
I20260808 10:12:09.072896 24823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:09.072961 24828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:12:09.073154 24820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:09.066581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46365 (local address 127.2.30.190:36377)
0808 10:12:09.066754 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:09.066764 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:09.066797 (+ 33us) server_negotiation.cc:407] Connection header received
0808 10:12:09.067533 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:09.067543 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:09.067604 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:09.067752 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:09.067937 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:09.069381 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:09.070338 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:09.071018 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:09.071130 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:09.072063 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:09.072067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:09.072068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:12:09.072358 (+ 290us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:09.072392 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:12:09.072404 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:09.072503 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:09.072825 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 10:12:09.072854 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260808 10:12:09.074255 24838 mvcc.cc:205] Tried to move back new op lower bound from 7316209373486882816 to 7316209373477793792. Current Snapshot: MvccSnapshot[applied={T|T < 7316209373486882816}]
I20260808 10:12:09.074276 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:12:09.074695 24828 catalog_manager.cc:1347] Generated new cluster ID: 3c330a4c331b4ea48c979a75ca24a372
I20260808 10:12:09.074744 24828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:12:09.083679 24828 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:12:09.084335 24828 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:12:09.085321 24820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:09.079395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54665 (local address 127.2.30.190:36377)
0808 10:12:09.079455 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:09.079457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:09.082112 (+ 2655us) server_negotiation.cc:407] Connection header received
0808 10:12:09.082265 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:09.082266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:09.082278 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:09.082362 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:09.082523 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:09.083071 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:09.084039 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:09.084252 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:09.084330 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:09.085034 (+ 704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:09.085036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:09.085036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:12:09.085040 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:09.085055 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:12:09.085059 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:09.085133 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:09.085215 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 10:12:09.085224 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 10:12:09.087905 24828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d17b09cb2804249813f257bed6da293: Generated new TSK 0
I20260808 10:12:09.088073 24828 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:12:09.093143 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 24742
W20260808 10:12:09.099694 24726 connection.cc:535] client connection to 127.2.30.190:36377 recv error: Network error: failed to read from TLS socket (remote: 127.2.30.190:36377): Connection reset by peer (error 104)
2026-08-08T10:12:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1373 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:12:09 dist-test-slave-w206 krb5kdc[24847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:12:09 dist-test-slave-w206 krb5kdc[24847](info): set up 2 sockets
Aug 08 10:12:09 dist-test-slave-w206 krb5kdc[24847](info): commencing operation
krb5kdc: starting...
W20260808 10:12:10.155582 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:12:10 dist-test-slave-w206 krb5kdc[24847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:12:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:12:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:12:10.290390 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:34753
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:33265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:34753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:12:10.379305 24860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:10.379536 24860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:10.379562 24860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:10.380805 24860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:10.380859 24860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:10.380875 24860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:10.380890 24860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:10.380904 24860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:10.382516 24860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:33265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:34753
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:34753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:10.382804 24860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:10.383104 24860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:12:10.388274 24866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:10.388306 24865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:10.388309 24868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:10.388749 24860 server_base.cc:1034] running on GCE node
I20260808 10:12:10.389034 24860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:10.389348 24860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:10.390522 24860 hybrid_clock.cc:648] HybridClock initialized: now 1786183930390495 us; error 41 us; skew 500 ppm
Aug 08 10:12:10 dist-test-slave-w206 krb5kdc[24847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183930, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:10.393354 24860 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:12:10.393862 24860 webserver.cc:460] Webserver started at http://127.2.30.190:45715/ using document root <none> and password file <none>
I20260808 10:12:10.394104 24860 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:10.394186 24860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:10.394343 24860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:10.395533 24860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "5e189156c145408eb8fb42325f334e46"
format_stamp: "Formatted at 2026-08-08 10:12:10 on dist-test-slave-w206"
server_key: "69b6127d483e0cb5dd26feab4ea291da"
server_key_iv: "5781cf44de1eca85284d034bf5ee59a0"
server_key_version: "encryptionkey@0"
I20260808 10:12:10.395912 24860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "5e189156c145408eb8fb42325f334e46"
format_stamp: "Formatted at 2026-08-08 10:12:10 on dist-test-slave-w206"
server_key: "69b6127d483e0cb5dd26feab4ea291da"
server_key_iv: "5781cf44de1eca85284d034bf5ee59a0"
server_key_version: "encryptionkey@0"
I20260808 10:12:10.397447 24860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 10:12:10.398315 24875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:10.398576 24860 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:12:10.398699 24860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "5e189156c145408eb8fb42325f334e46"
format_stamp: "Formatted at 2026-08-08 10:12:10 on dist-test-slave-w206"
server_key: "69b6127d483e0cb5dd26feab4ea291da"
server_key_iv: "5781cf44de1eca85284d034bf5ee59a0"
server_key_version: "encryptionkey@0"
I20260808 10:12:10.398797 24860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:10.402643 24860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:10.405254 24860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:10.405428 24860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:10.409829 24860 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:34753
I20260808 10:12:10.409893 24930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:34753 every 8 connection(s)
I20260808 10:12:10.410293 24860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:12:10.411515 24931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:10.414018 24931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46: Bootstrap starting.
I20260808 10:12:10.414603 24931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:10.414882 24931 log.cc:824] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:10.415680 24931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46: No bootstrap required, opened a new log
I20260808 10:12:10.416478 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 24860
I20260808 10:12:10.416632 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:12:10.416875 2170 external_mini_cluster.cc:1428] Setting key 439c38576214269ff70cd4816488bbf0
I20260808 10:12:10.417863 24931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e189156c145408eb8fb42325f334e46" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 34753 } }
I20260808 10:12:10.418048 24931 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:12:10.418092 24931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e189156c145408eb8fb42325f334e46, State: Initialized, Role: FOLLOWER
I20260808 10:12:10.418259 24931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e189156c145408eb8fb42325f334e46" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 34753 } }
I20260808 10:12:10.418362 24931 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:12:10.418397 24931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:12:10.418493 24931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:10.419536 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:10.417488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39105 (local address 127.2.30.190:34753)
0808 10:12:10.417750 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:10.417759 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:10.417800 (+ 41us) server_negotiation.cc:407] Connection header received
0808 10:12:10.417937 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:10.417945 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:10.417976 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:10.418130 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:10.418927 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:10.418930 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:10.418930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:12:10.418934 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:10.419033 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:12:10.419049 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:10.419131 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:10.419236 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 10:12:10.419283 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":121,"threads_started":1}
I20260808 10:12:10.419754 24931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e189156c145408eb8fb42325f334e46" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 34753 } }
I20260808 10:12:10.419867 24931 leader_election.cc:302] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e189156c145408eb8fb42325f334e46; no voters:
I20260808 10:12:10.420058 24931 leader_election.cc:288] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:12:10.420113 24936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:12:10.420326 24936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [term 1 LEADER]: Becoming Leader. State: Replica: 5e189156c145408eb8fb42325f334e46, State: Running, Role: LEADER
I20260808 10:12:10.420334 24931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:10.420511 24936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e189156c145408eb8fb42325f334e46" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 34753 } }
I20260808 10:12:10.420923 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:12:10.421132 24937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e189156c145408eb8fb42325f334e46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e189156c145408eb8fb42325f334e46" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 34753 } } }
I20260808 10:12:10.421277 24937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:10.421768 24945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:12:10.422467 24945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:12:10.422701 24938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e189156c145408eb8fb42325f334e46. Latest consensus state: current_term: 1 leader_uuid: "5e189156c145408eb8fb42325f334e46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e189156c145408eb8fb42325f334e46" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 34753 } } }
I20260808 10:12:10.422785 24938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:10.424217 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:10.422606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51171 (local address 127.2.30.190:34753)
0808 10:12:10.422642 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:10.422644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:10.422896 (+ 252us) server_negotiation.cc:407] Connection header received
0808 10:12:10.423028 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:10.423029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:10.423035 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:10.423112 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:10.423812 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:10.423813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:10.423814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:12:10.423818 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:10.423834 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:12:10.423839 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:10.423901 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:10.424136 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 10:12:10.424151 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 10:12:10.424835 24945 catalog_manager.cc:1347] Generated new cluster ID: 4592af914df843be937741e027efa423
I20260808 10:12:10.424899 24945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:12:10.432332 24945 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:12:10.432514 24945 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:12:10.444042 24945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e189156c145408eb8fb42325f334e46: Generated new TSK 0
I20260808 10:12:10.444274 24945 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:12:10.452062 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 24860
2026-08-08T10:12:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1358 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:12:10 dist-test-slave-w206 krb5kdc[24966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:12:10 dist-test-slave-w206 krb5kdc[24966](info): set up 2 sockets
Aug 08 10:12:10 dist-test-slave-w206 krb5kdc[24966](info): commencing operation
krb5kdc: starting...
W20260808 10:12:11.534652 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:12:11 dist-test-slave-w206 krb5kdc[24966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:12:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:12:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:12:11.668334 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:46081
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:35499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:46081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:12:11.752992 24979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:11.753300 24979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:11.753348 24979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:11.754607 24979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:11.754691 24979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:11.754717 24979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:11.754737 24979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:11.754756 24979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:11.756390 24979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:35499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:46081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:46081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:11.756702 24979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:11.756956 24979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:12:11.762276 24987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:11.762487 24984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:11.762276 24985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:11.762619 24979 server_base.cc:1034] running on GCE node
I20260808 10:12:11.762861 24979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:11.763247 24979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:11.764422 24979 hybrid_clock.cc:648] HybridClock initialized: now 1786183931764397 us; error 45 us; skew 500 ppm
Aug 08 10:12:11 dist-test-slave-w206 krb5kdc[24966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183931, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:11.767304 24979 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:12:11.767764 24979 webserver.cc:460] Webserver started at http://127.2.30.190:45739/ using document root <none> and password file <none>
I20260808 10:12:11.768016 24979 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:11.768088 24979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:11.768252 24979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:11.769407 24979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "e1b04045102d4d8194ebc9f25996048e"
format_stamp: "Formatted at 2026-08-08 10:12:11 on dist-test-slave-w206"
server_key: "b678976fd461a61ce1ac273663600ad6"
server_key_iv: "d91c406c06189958af1d9fdba874adb4"
server_key_version: "encryptionkey@0"
I20260808 10:12:11.769805 24979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "e1b04045102d4d8194ebc9f25996048e"
format_stamp: "Formatted at 2026-08-08 10:12:11 on dist-test-slave-w206"
server_key: "b678976fd461a61ce1ac273663600ad6"
server_key_iv: "d91c406c06189958af1d9fdba874adb4"
server_key_version: "encryptionkey@0"
I20260808 10:12:11.771564 24979 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 10:12:11.772476 24995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:11.772712 24979 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:12:11.772881 24979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "e1b04045102d4d8194ebc9f25996048e"
format_stamp: "Formatted at 2026-08-08 10:12:11 on dist-test-slave-w206"
server_key: "b678976fd461a61ce1ac273663600ad6"
server_key_iv: "d91c406c06189958af1d9fdba874adb4"
server_key_version: "encryptionkey@0"
I20260808 10:12:11.772982 24979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:11.784216 24979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:11.787555 24979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:11.787753 24979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:11.792369 24979 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:46081
I20260808 10:12:11.792444 25047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:46081 every 8 connection(s)
I20260808 10:12:11.792829 24979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:12:11.793808 25048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:11.794221 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 24979
I20260808 10:12:11.794360 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:12:11.794721 2170 external_mini_cluster.cc:1428] Setting key 9c52bd45fe4b8c36cb860d1c494a20fc
I20260808 10:12:11.797163 25048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e: Bootstrap starting.
I20260808 10:12:11.797914 25048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:11.798252 25048 log.cc:824] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:11.799103 25048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e: No bootstrap required, opened a new log
I20260808 10:12:11.801232 25048 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1b04045102d4d8194ebc9f25996048e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46081 } }
I20260808 10:12:11.801390 25048 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:12:11.801414 25048 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1b04045102d4d8194ebc9f25996048e, State: Initialized, Role: FOLLOWER
I20260808 10:12:11.801554 25048 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1b04045102d4d8194ebc9f25996048e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46081 } }
I20260808 10:12:11.801638 25048 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:12:11.801664 25048 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:12:11.801728 25048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:11.802377 25048 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1b04045102d4d8194ebc9f25996048e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46081 } }
I20260808 10:12:11.802510 25048 leader_election.cc:302] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1b04045102d4d8194ebc9f25996048e; no voters:
I20260808 10:12:11.802753 25048 leader_election.cc:288] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:12:11.802795 25054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [term 1 FOLLOWER]: Leader election won for term 1
Aug 08 10:12:11 dist-test-slave-w206 krb5kdc[24966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:12:11.802917 25054 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [term 1 LEADER]: Becoming Leader. State: Replica: e1b04045102d4d8194ebc9f25996048e, State: Running, Role: LEADER
I20260808 10:12:11.803025 25054 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1b04045102d4d8194ebc9f25996048e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46081 } }
I20260808 10:12:11.803074 25048 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:11.803391 25054 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1b04045102d4d8194ebc9f25996048e. Latest consensus state: current_term: 1 leader_uuid: "e1b04045102d4d8194ebc9f25996048e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1b04045102d4d8194ebc9f25996048e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46081 } } }
I20260808 10:12:11.803488 25054 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:11.803751 25059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:12:11.804337 25055 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1b04045102d4d8194ebc9f25996048e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1b04045102d4d8194ebc9f25996048e" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46081 } } }
I20260808 10:12:11.804600 25055 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:11.804924 25059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:12:11.807106 25059 catalog_manager.cc:1347] Generated new cluster ID: e4e2e35bf5e64fe3aac009cdce1c053e
I20260808 10:12:11.807245 25059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:12:11.808722 25051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:11.795379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42287 (local address 127.2.30.190:46081)
0808 10:12:11.795586 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:11.795598 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:11.795637 (+ 39us) server_negotiation.cc:407] Connection header received
0808 10:12:11.796187 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:11.796209 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:11.796273 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:11.796437 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:11.797142 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:11.798569 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:11.799540 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:11.800017 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:11.800161 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:11.803886 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:11.803891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:11.803891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:11.803908 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:11.806664 (+ 2756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:11.807239 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:11.807242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:11.807243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:11.807386 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:11.807671 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:11.807671 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:11.807672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:11.808018 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:11.808176 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:11.808318 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 10:12:11.808494 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":87,"threads_started":1}
I20260808 10:12:11.809758 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:12:11.828085 25051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:11.811481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59847 (local address 127.2.30.190:46081)
0808 10:12:11.811522 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:11.811524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:11.819410 (+ 7886us) server_negotiation.cc:407] Connection header received
0808 10:12:11.819500 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:11.819501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:11.819516 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:11.819600 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:11.820480 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:11.821317 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:11.822493 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:11.822865 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:11.822945 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:11.824768 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:11.824770 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:11.824771 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:11.824779 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:11.826639 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:11.827150 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:11.827152 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:11.827152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:11.827221 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:11.827462 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:11.827463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:11.827463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:11.827614 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:11.827716 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:11.827881 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 10:12:11.827976 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:12:11.837337 25059 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:12:11.838056 25059 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:12:11.844913 25059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1b04045102d4d8194ebc9f25996048e: Generated new TSK 0
I20260808 10:12:11.845155 25059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:12:11.847523 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 24979
2026-08-08T10:12:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1396 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:12:11 dist-test-slave-w206 krb5kdc[25084](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:12:11 dist-test-slave-w206 krb5kdc[25084](info): set up 2 sockets
Aug 08 10:12:11 dist-test-slave-w206 krb5kdc[25084](info): commencing operation
krb5kdc: starting...
W20260808 10:12:12.940397 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:12:13 dist-test-slave-w206 krb5kdc[25084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:12:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:12:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:12:13.078896 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:41605
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:33475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:41605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:12:13.165097 25099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:13.165341 25099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:13.165367 25099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:13.166620 25099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:13.166671 25099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:13.166685 25099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:13.166698 25099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:13.166711 25099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:13.168340 25099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:33475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:41605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:41605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:13.168642 25099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:13.168910 25099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 10:12:13.174558 25099 server_base.cc:1034] running on GCE node
W20260808 10:12:13.174506 25104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:13.174518 25107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:13.174506 25105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:13.175148 25099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:13.175475 25099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:13.176659 25099 hybrid_clock.cc:648] HybridClock initialized: now 1786183933176643 us; error 34 us; skew 500 ppm
Aug 08 10:12:13 dist-test-slave-w206 krb5kdc[25084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183933, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:13.194535 25099 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:12:13.195163 25099 webserver.cc:460] Webserver started at http://127.2.30.190:45487/ using document root <none> and password file <none>
I20260808 10:12:13.195436 25099 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:13.195505 25099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:13.195672 25099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:13.196813 25099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "bbc3d596eb9446f295e31d10f1d3fef7"
format_stamp: "Formatted at 2026-08-08 10:12:13 on dist-test-slave-w206"
server_key: "881fe3593f2b17a7882d0ca44050f401"
server_key_iv: "04d66a35ac90c5648521438eb6edb98e"
server_key_version: "encryptionkey@0"
I20260808 10:12:13.197198 25099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "bbc3d596eb9446f295e31d10f1d3fef7"
format_stamp: "Formatted at 2026-08-08 10:12:13 on dist-test-slave-w206"
server_key: "881fe3593f2b17a7882d0ca44050f401"
server_key_iv: "04d66a35ac90c5648521438eb6edb98e"
server_key_version: "encryptionkey@0"
I20260808 10:12:13.198674 25099 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:12:13.199504 25114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:13.199676 25099 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:13.199801 25099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "bbc3d596eb9446f295e31d10f1d3fef7"
format_stamp: "Formatted at 2026-08-08 10:12:13 on dist-test-slave-w206"
server_key: "881fe3593f2b17a7882d0ca44050f401"
server_key_iv: "04d66a35ac90c5648521438eb6edb98e"
server_key_version: "encryptionkey@0"
I20260808 10:12:13.199882 25099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:13.212446 25099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:13.214675 25099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:13.214890 25099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:13.219245 25099 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:41605
I20260808 10:12:13.219295 25168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:41605 every 8 connection(s)
I20260808 10:12:13.219664 25099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:12:13.220579 25169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:13.223217 25169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7: Bootstrap starting.
I20260808 10:12:13.223836 25169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:13.224154 25169 log.cc:824] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:13.224959 25169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7: No bootstrap required, opened a new log
I20260808 10:12:13.225787 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 25099
I20260808 10:12:13.225881 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:12:13.226122 2170 external_mini_cluster.cc:1428] Setting key a235c97315013d8da207268e6a7ade2b
I20260808 10:12:13.227157 25169 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc3d596eb9446f295e31d10f1d3fef7" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41605 } }
I20260808 10:12:13.227340 25169 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:12:13.227408 25169 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbc3d596eb9446f295e31d10f1d3fef7, State: Initialized, Role: FOLLOWER
I20260808 10:12:13.227560 25169 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc3d596eb9446f295e31d10f1d3fef7" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41605 } }
I20260808 10:12:13.227666 25169 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:12:13.227706 25169 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:12:13.227761 25169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:13.228386 25169 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc3d596eb9446f295e31d10f1d3fef7" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41605 } }
I20260808 10:12:13.228513 25169 leader_election.cc:302] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbc3d596eb9446f295e31d10f1d3fef7; no voters:
I20260808 10:12:13.228717 25169 leader_election.cc:288] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:12:13.228753 25174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:12:13.228914 25174 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [term 1 LEADER]: Becoming Leader. State: Replica: bbc3d596eb9446f295e31d10f1d3fef7, State: Running, Role: LEADER
I20260808 10:12:13.229012 25169 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:13.229050 25174 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc3d596eb9446f295e31d10f1d3fef7" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41605 } }
I20260808 10:12:13.230062 25176 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbc3d596eb9446f295e31d10f1d3fef7. Latest consensus state: current_term: 1 leader_uuid: "bbc3d596eb9446f295e31d10f1d3fef7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc3d596eb9446f295e31d10f1d3fef7" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41605 } } }
I20260808 10:12:13.230161 25176 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:13.230180 25175 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbc3d596eb9446f295e31d10f1d3fef7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbc3d596eb9446f295e31d10f1d3fef7" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41605 } } }
I20260808 10:12:13.230260 25175 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7 [sys.catalog]: This master's current role is: LEADER
W20260808 10:12:13.230988 25191 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:12:13.231606 25191 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:12:13.231693 25188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:12:13.231907 25188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:12:13.232460 25173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:13.226705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44671 (local address 127.2.30.190:41605)
0808 10:12:13.226959 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:13.226968 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:13.227043 (+ 75us) server_negotiation.cc:407] Connection header received
0808 10:12:13.227155 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:13.227161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:13.227220 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:13.227367 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:13.227602 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:13.229028 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:13.230214 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:13.230781 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:13.230922 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:13.231769 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:13.231773 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:13.231774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:12:13.231991 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:13.232032 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:12:13.232048 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:13.232120 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:13.232183 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 10:12:13.232246 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":123,"threads_started":1}
I20260808 10:12:13.233546 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:12:13.233583 25188 catalog_manager.cc:1347] Generated new cluster ID: a0e486c2b9004b1abd21fee0faed06df
I20260808 10:12:13.233628 25188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:12:13.242659 25173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:13.235196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52305 (local address 127.2.30.190:41605)
0808 10:12:13.235234 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:13.235236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:13.239225 (+ 3989us) server_negotiation.cc:407] Connection header received
0808 10:12:13.239310 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:13.239311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:13.239322 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:13.239382 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:13.239519 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:13.240238 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:13.241279 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:13.241582 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:13.241640 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:13.242397 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:13.242398 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:13.242398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:12:13.242403 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:13.242417 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:12:13.242421 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:13.242465 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:13.242549 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 10:12:13.242573 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:12:13.258761 25188 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:12:13.259436 25188 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:12:13.267964 25188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbc3d596eb9446f295e31d10f1d3fef7: Generated new TSK 0
I20260808 10:12:13.268188 25188 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:12:13.271474 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 25099
2026-08-08T10:12:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:12:13 dist-test-slave-w206 krb5kdc[25201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:12:13 dist-test-slave-w206 krb5kdc[25201](info): set up 2 sockets
Aug 08 10:12:13 dist-test-slave-w206 krb5kdc[25201](info): commencing operation
krb5kdc: starting...
W20260808 10:12:14.384147 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:12:14 dist-test-slave-w206 krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:12:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:12:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:12:14.522197 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:46255
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:36309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:46255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:12:14.607514 25215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:14.607788 25215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:14.607828 25215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:14.609037 25215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:14.609115 25215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:14.609143 25215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:14.609175 25215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:14.609201 25215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:14.610719 25215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:36309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:46255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:46255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:14.611052 25215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:14.611317 25215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:12:14.616961 25223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:14.616951 25221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:14.616951 25220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:14.616989 25215 server_base.cc:1034] running on GCE node
I20260808 10:12:14.617475 25215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:14.617862 25215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:14.619062 25215 hybrid_clock.cc:648] HybridClock initialized: now 1786183934619043 us; error 50 us; skew 500 ppm
Aug 08 10:12:14 dist-test-slave-w206 krb5kdc[25201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183934, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:14.622123 25215 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:12:14.622709 25215 webserver.cc:460] Webserver started at http://127.2.30.190:42325/ using document root <none> and password file <none>
I20260808 10:12:14.623067 25215 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:14.623148 25215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:14.623332 25215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:14.624524 25215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "332016dbed4e447e8dbf40ea4078ab51"
format_stamp: "Formatted at 2026-08-08 10:12:14 on dist-test-slave-w206"
server_key: "04224e995ff073abb134ac4b1deb3773"
server_key_iv: "a68f0b6d9d939155f9681abd88995cce"
server_key_version: "encryptionkey@0"
I20260808 10:12:14.624929 25215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "332016dbed4e447e8dbf40ea4078ab51"
format_stamp: "Formatted at 2026-08-08 10:12:14 on dist-test-slave-w206"
server_key: "04224e995ff073abb134ac4b1deb3773"
server_key_iv: "a68f0b6d9d939155f9681abd88995cce"
server_key_version: "encryptionkey@0"
I20260808 10:12:14.626508 25215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:12:14.627465 25230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:14.627786 25215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:12:14.627907 25215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "332016dbed4e447e8dbf40ea4078ab51"
format_stamp: "Formatted at 2026-08-08 10:12:14 on dist-test-slave-w206"
server_key: "04224e995ff073abb134ac4b1deb3773"
server_key_iv: "a68f0b6d9d939155f9681abd88995cce"
server_key_version: "encryptionkey@0"
I20260808 10:12:14.627991 25215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:14.631923 25215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:14.634218 25215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:14.634444 25215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:14.638775 25215 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:46255
I20260808 10:12:14.638839 25284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:46255 every 8 connection(s)
I20260808 10:12:14.639219 25215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:12:14.640606 25285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:14.643288 25285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51: Bootstrap starting.
I20260808 10:12:14.643945 25285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:14.644270 25285 log.cc:824] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:14.645134 25285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51: No bootstrap required, opened a new log
I20260808 10:12:14.647138 25285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332016dbed4e447e8dbf40ea4078ab51" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46255 } }
I20260808 10:12:14.647291 25285 raft_consensus.cc:374] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 10:12:14.647317 25285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 332016dbed4e447e8dbf40ea4078ab51, State: Initialized, Role: FOLLOWER
I20260808 10:12:14.647537 25285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332016dbed4e447e8dbf40ea4078ab51" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46255 } }
I20260808 10:12:14.647609 25285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 10:12:14.647634 25285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 10:12:14.647665 25285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:14.648613 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 25215
I20260808 10:12:14.648764 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:12:14.648742 25285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332016dbed4e447e8dbf40ea4078ab51" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46255 } }
I20260808 10:12:14.648885 25285 leader_election.cc:302] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 332016dbed4e447e8dbf40ea4078ab51; no voters:
I20260808 10:12:14.648985 2170 external_mini_cluster.cc:1428] Setting key 2e0864b375da59819b1e866137c11d59
I20260808 10:12:14.649089 25285 leader_election.cc:288] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 10:12:14.649123 25290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:12:14.649261 25290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [term 1 LEADER]: Becoming Leader. State: Replica: 332016dbed4e447e8dbf40ea4078ab51, State: Running, Role: LEADER
I20260808 10:12:14.649375 25290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332016dbed4e447e8dbf40ea4078ab51" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46255 } }
I20260808 10:12:14.649772 25290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "332016dbed4e447e8dbf40ea4078ab51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332016dbed4e447e8dbf40ea4078ab51" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46255 } } }
I20260808 10:12:14.649781 25291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 332016dbed4e447e8dbf40ea4078ab51. Latest consensus state: current_term: 1 leader_uuid: "332016dbed4e447e8dbf40ea4078ab51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "332016dbed4e447e8dbf40ea4078ab51" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 46255 } } }
I20260808 10:12:14.649883 25285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:14.649935 25290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:14.649961 25291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:14.650293 25296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:12:14.651398 25296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:12:14.651908 25293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:14.649532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59199 (local address 127.2.30.190:46255)
0808 10:12:14.649940 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:14.649950 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:14.649987 (+ 37us) server_negotiation.cc:407] Connection header received
0808 10:12:14.650113 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:14.650122 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:14.650153 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:14.650647 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:14.651269 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:14.651273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:14.651273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:12:14.651277 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:14.651318 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:12:14.651335 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:14.651437 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:14.651562 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 10:12:14.651606 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":92,"threads_started":1}
I20260808 10:12:14.652908 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:12:14.653246 25296 catalog_manager.cc:1347] Generated new cluster ID: f6c4712c15c7400cad686a6d7433e255
I20260808 10:12:14.653306 25296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:12:14.655685 25293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:14.654481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49241 (local address 127.2.30.190:46255)
0808 10:12:14.654518 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:14.654519 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:14.654667 (+ 148us) server_negotiation.cc:407] Connection header received
0808 10:12:14.654714 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:14.654715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:14.654732 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:14.654792 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:14.655470 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:14.655471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:14.655471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 10:12:14.655475 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:14.655490 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 10:12:14.655494 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:14.655541 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:14.655595 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 10:12:14.655610 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:12:14.668629 25296 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:12:14.668859 25296 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:12:14.672034 25296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 332016dbed4e447e8dbf40ea4078ab51: Generated new TSK 0
I20260808 10:12:14.672307 25296 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 10:12:14.675047 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 25215
2026-08-08T10:12:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1402 ms)
[----------] 18 tests from AuthTokenIssuingTest (25237 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:12:14 dist-test-slave-w206 krb5kdc[25320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:12:14 dist-test-slave-w206 krb5kdc[25320](info): set up 2 sockets
Aug 08 10:12:14 dist-test-slave-w206 krb5kdc[25320](info): commencing operation
krb5kdc: starting...
W20260808 10:12:15.751699 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:12:15 dist-test-slave-w206 krb5kdc[25320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:12:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:12:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:12:15.887447 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:37951
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:42165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:37951,127.2.30.189:36443,127.2.30.188:35697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:12:15.974172 25333 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:12:15.974448 25333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:15.974479 25333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:15.974527 25333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:15.975766 25333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:15.975849 25333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:15.975873 25333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:15.975891 25333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:15.975922 25333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:15.977573 25333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:42165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:37951,127.2.30.189:36443,127.2.30.188:35697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:37951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:15.977900 25333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:15.978204 25333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:12:15.983609 25338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:15.983568 25341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:15.983570 25339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:15.983955 25333 server_base.cc:1034] running on GCE node
I20260808 10:12:15.984315 25333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:15.984651 25333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:15.985831 25333 hybrid_clock.cc:648] HybridClock initialized: now 1786183935985810 us; error 46 us; skew 500 ppm
Aug 08 10:12:15 dist-test-slave-w206 krb5kdc[25320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183935, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:15.988773 25333 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:12:15.989317 25333 webserver.cc:460] Webserver started at http://127.2.30.190:45431/ using document root <none> and password file <none>
I20260808 10:12:15.989562 25333 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:15.989631 25333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:15.989790 25333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:15.991030 25333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "a0e7bdd3c06a4044a8e006e2af42769f"
format_stamp: "Formatted at 2026-08-08 10:12:15 on dist-test-slave-w206"
server_key: "f3cbb7ba331962de414ddb5c6a7dbee1"
server_key_iv: "eeff95ae914d67f0fbc162c49dc2042d"
server_key_version: "encryptionkey@0"
I20260808 10:12:15.991444 25333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "a0e7bdd3c06a4044a8e006e2af42769f"
format_stamp: "Formatted at 2026-08-08 10:12:15 on dist-test-slave-w206"
server_key: "f3cbb7ba331962de414ddb5c6a7dbee1"
server_key_iv: "eeff95ae914d67f0fbc162c49dc2042d"
server_key_version: "encryptionkey@0"
I20260808 10:12:15.992930 25333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:12:15.993821 25349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:15.994012 25333 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:12:15.994124 25333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "a0e7bdd3c06a4044a8e006e2af42769f"
format_stamp: "Formatted at 2026-08-08 10:12:15 on dist-test-slave-w206"
server_key: "f3cbb7ba331962de414ddb5c6a7dbee1"
server_key_iv: "eeff95ae914d67f0fbc162c49dc2042d"
server_key_version: "encryptionkey@0"
I20260808 10:12:15.994203 25333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:16.021190 25333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:16.023608 25333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:16.023794 25333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:16.028342 25333 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:37951
I20260808 10:12:16.028412 25401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:37951 every 8 connection(s)
I20260808 10:12:16.028788 25333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:12:16.029783 25402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:16.031405 25402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:12:16.034260 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 25333
I20260808 10:12:16.034416 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:12:16.034619 2170 external_mini_cluster.cc:1428] Setting key d9e19d90193348f46b67f176405794cb
Aug 08 10:12:16 dist-test-slave-w206 krb5kdc[25320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183935, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:12:16.042313 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.032051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:37951 (local address 127.0.0.1:51836)
0808 10:12:16.032270 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.032324 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:16.032811 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:16.033103 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:16.033106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:16.033444 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:16.033600 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.033604 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.034767 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.034768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.036005 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.036008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.036566 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.036567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.036673 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.037284 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:16.037292 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:16.038717 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:16.040711 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.040718 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.040720 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.040929 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.041198 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.041199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.041199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.041295 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.041688 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:16.041693 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:16.041759 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 10:12:16.041864 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 10:12:16.042119 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":106,"thread_start_us":92,"threads_started":1}
I20260808 10:12:16.042311 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.032113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51836 (local address 127.2.30.190:37951)
0808 10:12:16.032281 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.032290 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.032427 (+ 137us) server_negotiation.cc:407] Connection header received
0808 10:12:16.032976 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.032979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.032996 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.033097 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.033706 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.034642 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.036104 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.036414 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.036560 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.038816 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:16.038825 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:16.038825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:16.038832 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:16.040597 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.041033 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.041037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.041037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.041126 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.041373 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.041375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.041375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.041604 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:16.041697 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:16.041908 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 10:12:16.042052 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for kudu/127.2.30.189@KRBTEST.COM; defaulting to no policy
I20260808 10:12:16.043138 25402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:12:16.043530 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.043424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:36443 (local address 127.0.0.1:48996)
0808 10:12:16.043464 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.043510 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:36443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 10:12:16.043681 25351 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.30.189:36443: connect: Connection refused (error 111)
I20260808 10:12:16.044262 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.044168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:36443 (local address 127.0.0.1:49006)
0808 10:12:16.044220 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.044245 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:36443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 10:12:16.044443 25402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.189:36443: Network error: Client connection negotiation failed: client connection to 127.2.30.189:36443: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.189.keytab.
Entry for principal kudu/127.2.30.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.189.keytab.
I20260808 10:12:16.062968 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.189:36443
--webserver_interface=127.2.30.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.189.keytab
--principal=kudu/127.2.30.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:42165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:37951,127.2.30.189:36443,127.2.30.188:35697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:12:16.093683 25402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } attempt: 1
I20260808 10:12:16.094260 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.094081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:36443 (local address 127.0.0.1:49012)
0808 10:12:16.094155 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.094222 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:36443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 10:12:16.094630 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.094529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:36443 (local address 127.0.0.1:49018)
0808 10:12:16.094578 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.094604 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:36443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 10:12:16.094725 25402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.189:36443: Network error: Client connection negotiation failed: client connection to 127.2.30.189:36443: connect: Connection refused (error 111)
W20260808 10:12:16.149331 25409 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:12:16.149607 25409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:16.149641 25409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:16.149689 25409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:16.150897 25409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:16.150981 25409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:16.151031 25409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:16.151064 25409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:16.151086 25409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:16.152705 25409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:42165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:37951,127.2.30.189:36443,127.2.30.188:35697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.189.keytab
--principal=kudu/127.2.30.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.189:36443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.30.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:16.153040 25409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:16.153303 25409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:12:16.158576 25418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:16.158655 25415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:16.158584 25416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:16.158830 25409 server_base.cc:1034] running on GCE node
I20260808 10:12:16.159236 25409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:16.159587 25409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:16.160816 25409 hybrid_clock.cc:648] HybridClock initialized: now 1786183936160796 us; error 50 us; skew 500 ppm
I20260808 10:12:16.162889 25402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } attempt: 2
Aug 08 10:12:16 dist-test-slave-w206 krb5kdc[25320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183936, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:16.163503 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.163344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:36443 (local address 127.0.0.1:49034)
0808 10:12:16.163427 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.163469 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:36443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 10:12:16.163959 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.163867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:36443 (local address 127.0.0.1:49050)
0808 10:12:16.163913 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.163940 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:36443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 10:12:16.163991 25409 init.cc:377] Logged in from keytab as kudu/127.2.30.189@KRBTEST.COM (short username kudu)
W20260808 10:12:16.164122 25402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.189:36443: Network error: Client connection negotiation failed: client connection to 127.2.30.189:36443: connect: Connection refused (error 111)
I20260808 10:12:16.164608 25409 webserver.cc:460] Webserver started at http://127.2.30.189:33203/ using document root <none> and password file <none>
I20260808 10:12:16.164888 25409 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:16.164939 25409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:16.165076 25409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:16.166201 25409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/data/instance:
uuid: "c1d92b5ab0d14b7aa930113184807c25"
format_stamp: "Formatted at 2026-08-08 10:12:16 on dist-test-slave-w206"
server_key: "07d7782723b44e63924fb1e9a2e3695c"
server_key_iv: "4af6e4b00c01fe4b674d162d8b2e9314"
server_key_version: "encryptionkey@0"
I20260808 10:12:16.166567 25409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/wal/instance:
uuid: "c1d92b5ab0d14b7aa930113184807c25"
format_stamp: "Formatted at 2026-08-08 10:12:16 on dist-test-slave-w206"
server_key: "07d7782723b44e63924fb1e9a2e3695c"
server_key_iv: "4af6e4b00c01fe4b674d162d8b2e9314"
server_key_version: "encryptionkey@0"
I20260808 10:12:16.168135 25409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:12:16.169055 25427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:16.169241 25409 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:12:16.169341 25409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/wal
uuid: "c1d92b5ab0d14b7aa930113184807c25"
format_stamp: "Formatted at 2026-08-08 10:12:16 on dist-test-slave-w206"
server_key: "07d7782723b44e63924fb1e9a2e3695c"
server_key_iv: "4af6e4b00c01fe4b674d162d8b2e9314"
server_key_version: "encryptionkey@0"
I20260808 10:12:16.169423 25409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:16.196398 25409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:16.199062 25409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:16.199196 25409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:16.203474 25409 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.189:36443
I20260808 10:12:16.203536 25480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.189:36443 every 8 connection(s)
I20260808 10:12:16.203912 25409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/data/info.pb
I20260808 10:12:16.204983 25481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:16.206590 25481 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:12:16.209168 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 25409
I20260808 10:12:16.209251 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-1/wal/instance
I20260808 10:12:16.209492 2170 external_mini_cluster.cc:1428] Setting key 2dfd520d099e6449b8659bc388c94376
Aug 08 10:12:16 dist-test-slave-w206 krb5kdc[25320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183936, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.189@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:12:16.217439 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.207353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51842 (local address 127.2.30.190:37951)
0808 10:12:16.207419 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.207422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.207608 (+ 186us) server_negotiation.cc:407] Connection header received
0808 10:12:16.208055 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.208056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.208068 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.208133 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.208770 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.209344 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.210868 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.211162 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.211231 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.214096 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:16.214099 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:16.214099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:16.214109 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:16.215807 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.216392 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.216393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.216394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.216456 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.216756 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.216756 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.216756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.216872 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:16.216923 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:16.217283 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 10:12:16.217368 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 10:12:16.217742 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.207293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:37951 (local address 127.0.0.1:51842)
0808 10:12:16.207474 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.207517 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:16.207975 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:16.208213 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:16.208216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:16.208501 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:16.208697 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.208700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.209448 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.209449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.210752 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.210756 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.211268 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.211269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.211460 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.212127 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:16.212136 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:16.213974 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:16.215914 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.215926 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.215929 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.216305 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.216528 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.216529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.216530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.216688 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.216938 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:16.216946 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:16.217128 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 10:12:16.217261 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 10:12:16.217461 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":91,"threads_started":1}
I20260808 10:12:16.218480 25481 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.30.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.188@KRBTEST.COM" created.
Aug 08 10:12:16 dist-test-slave-w206 krb5kdc[25320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183936, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.189@KRBTEST.COM for kudu/127.2.30.189@KRBTEST.COM
I20260808 10:12:16.226498 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.218775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:36443 (local address 127.0.0.1:49062)
0808 10:12:16.219049 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.219061 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:16.219105 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:16.219250 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:16.219251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:16.219420 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:16.219533 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.219535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.220480 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.220481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.220986 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.220987 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.221586 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.221586 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.221642 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.222108 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:16.222113 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:16.223421 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:16.225381 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.225382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.225383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.225558 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.225785 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.225786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.225786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.225842 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.226201 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:16.226202 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:16.226242 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 10:12:16.226311 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 10:12:16.226413 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254}
I20260808 10:12:16.226691 25485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.218866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49062 (local address 127.2.30.189:36443)
0808 10:12:16.219086 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.219090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.219107 (+ 17us) server_negotiation.cc:407] Connection header received
0808 10:12:16.219168 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.219169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.219181 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.219238 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.219606 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.220391 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.221055 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.221514 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.221573 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.223524 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:16.223534 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:16.223534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:16.223541 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:16.225271 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.225645 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.225648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.225649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.225716 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.225904 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.225904 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.225905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.226114 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:16.226201 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:16.226362 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 10:12:16.226488 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":183,"thread_start_us":142,"threads_started":1}
I20260808 10:12:16.227420 25481 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:12:16.227814 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.227673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:35697 (local address 127.0.0.1:56302)
0808 10:12:16.227724 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.227776 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 10:12:16.227988 25431 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
I20260808 10:12:16.228438 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.228342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:35697 (local address 127.0.0.1:56318)
0808 10:12:16.228395 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.228420 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 10:12:16.228596 25481 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:35697: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.188.keytab.
Entry for principal kudu/127.2.30.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.188.keytab.
I20260808 10:12:16.243378 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.188:35697
--webserver_interface=127.2.30.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.188.keytab
--principal=kudu/127.2.30.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:42165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:37951,127.2.30.189:36443,127.2.30.188:35697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:12:16.255287 25402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } attempt: 3
Aug 08 10:12:16 dist-test-slave-w206 krb5kdc[25320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183935, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for kudu/127.2.30.189@KRBTEST.COM
I20260808 10:12:16.262894 25485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.255768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49070 (local address 127.2.30.189:36443)
0808 10:12:16.255859 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.255862 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.255897 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:12:16.255932 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.255933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.255945 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.256009 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.256383 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.256893 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.257633 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.257862 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.257946 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.260228 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:16.260229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:16.260230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:16.260241 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:16.261684 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.262062 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.262064 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.262064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.262126 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.262332 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.262333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.262334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.262458 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:16.262535 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:16.262701 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 10:12:16.262787 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 10:12:16.262894 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.255705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:36443 (local address 127.0.0.1:49070)
0808 10:12:16.255776 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.255802 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:16.255856 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:16.256029 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:16.256031 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:16.256206 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:16.256276 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.256278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.256983 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.256984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.257539 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.257542 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.257945 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.257945 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.258029 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.258565 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:16.258569 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:16.260131 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:16.261787 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.261788 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.261789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.261970 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.262194 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.262195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.262196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.262258 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.262537 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:16.262539 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:16.262589 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 10:12:16.262676 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 10:12:16.262785 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 10:12:16.263813 25402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:12:16.264206 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.264101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:35697 (local address 127.0.0.1:56320)
0808 10:12:16.264154 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.264186 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 10:12:16.264525 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.264437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:35697 (local address 127.0.0.1:56330)
0808 10:12:16.264481 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.264504 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 10:12:16.264621 25402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:35697: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
I20260808 10:12:16.277837 25481 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } attempt: 1
I20260808 10:12:16.278308 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.278177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:35697 (local address 127.0.0.1:56334)
0808 10:12:16.278239 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.278279 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 10:12:16.278651 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.278557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:35697 (local address 127.0.0.1:56336)
0808 10:12:16.278602 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.278628 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 10:12:16.278731 25481 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:35697: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
I20260808 10:12:16.295774 25402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } attempt: 1
I20260808 10:12:16.296337 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.296158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:35697 (local address 127.0.0.1:56344)
0808 10:12:16.296223 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.296265 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 10:12:16.296700 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.296601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:35697 (local address 127.0.0.1:56346)
0808 10:12:16.296647 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.296672 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 10:12:16.296823 25402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:35697: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
W20260808 10:12:16.331719 25488 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:12:16.331984 25488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:16.332015 25488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:16.332062 25488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:16.333235 25488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:16.333314 25488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:16.333340 25488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:16.333396 25488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:16.333426 25488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:16.335100 25488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:42165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:37951,127.2.30.189:36443,127.2.30.188:35697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.188.keytab
--principal=kudu/127.2.30.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.188:35697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.30.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:16.335425 25488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:16.335686 25488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:12:16.341147 25494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:16.341156 25493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:16.341156 25496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:16.341172 25488 server_base.cc:1034] running on GCE node
I20260808 10:12:16.341789 25488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:16.342104 25488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:16.343247 25488 hybrid_clock.cc:648] HybridClock initialized: now 1786183936343214 us; error 48 us; skew 500 ppm
Aug 08 10:12:16 dist-test-slave-w206 krb5kdc[25320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183936, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:16.345904 25488 init.cc:377] Logged in from keytab as kudu/127.2.30.188@KRBTEST.COM (short username kudu)
I20260808 10:12:16.346396 25488 webserver.cc:460] Webserver started at http://127.2.30.188:36553/ using document root <none> and password file <none>
I20260808 10:12:16.346606 25488 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:16.346647 25488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:16.346742 25488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:16.346880 25481 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } attempt: 2
I20260808 10:12:16.347381 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.347225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:35697 (local address 127.0.0.1:56354)
0808 10:12:16.347294 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.347336 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 10:12:16.347802 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.347692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:35697 (local address 127.0.0.1:56358)
0808 10:12:16.347742 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.347772 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 10:12:16.347942 25481 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:35697: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
I20260808 10:12:16.348260 25488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/data/instance:
uuid: "722e50e9077e4f6e9eac075422818a74"
format_stamp: "Formatted at 2026-08-08 10:12:16 on dist-test-slave-w206"
server_key: "ee45940d3c571bf9617342abc9a5cf47"
server_key_iv: "de388d1ce0baa0bfe404bd56eb39038c"
server_key_version: "encryptionkey@0"
I20260808 10:12:16.348639 25488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/wal/instance:
uuid: "722e50e9077e4f6e9eac075422818a74"
format_stamp: "Formatted at 2026-08-08 10:12:16 on dist-test-slave-w206"
server_key: "ee45940d3c571bf9617342abc9a5cf47"
server_key_iv: "de388d1ce0baa0bfe404bd56eb39038c"
server_key_version: "encryptionkey@0"
I20260808 10:12:16.350543 25488 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 10:12:16.351680 25504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:16.351927 25488 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 10:12:16.352077 25488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/wal
uuid: "722e50e9077e4f6e9eac075422818a74"
format_stamp: "Formatted at 2026-08-08 10:12:16 on dist-test-slave-w206"
server_key: "ee45940d3c571bf9617342abc9a5cf47"
server_key_iv: "de388d1ce0baa0bfe404bd56eb39038c"
server_key_version: "encryptionkey@0"
I20260808 10:12:16.352173 25488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:16.371976 25402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } attempt: 2
I20260808 10:12:16.372646 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.372509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:35697 (local address 127.0.0.1:56364)
0808 10:12:16.372575 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.372616 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 10:12:16.373077 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.372971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:35697 (local address 127.0.0.1:56374)
0808 10:12:16.373025 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.373058 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 10:12:16.373204 25402 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:35697: Network error: Client connection negotiation failed: client connection to 127.2.30.188:35697: connect: Connection refused (error 111)
I20260808 10:12:16.376911 25488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:16.380373 25488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:16.380563 25488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:16.386564 25488 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.188:35697
I20260808 10:12:16.386617 25560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.188:35697 every 8 connection(s)
I20260808 10:12:16.387195 25488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/data/info.pb
I20260808 10:12:16.388306 25561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:16.389707 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 25488
I20260808 10:12:16.389823 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786183651681294-2170-0/minicluster-data/master-2/wal/instance
I20260808 10:12:16.390048 2170 external_mini_cluster.cc:1428] Setting key c46fbe27167d31d34b596881e38fe56d
I20260808 10:12:16.390244 25561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:12:16 dist-test-slave-w206 krb5kdc[25320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183936, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.188@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
Aug 08 10:12:16 dist-test-slave-w206 krb5kdc[25320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.188@KRBTEST.COM
I20260808 10:12:16.401093 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.390935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51844 (local address 127.2.30.190:37951)
0808 10:12:16.390977 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.390979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.391270 (+ 291us) server_negotiation.cc:407] Connection header received
0808 10:12:16.391978 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.391980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.391993 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.392046 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.392910 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.393392 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.394324 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.394602 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.394675 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.397464 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:16.397466 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:16.397466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:16.397474 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:16.399422 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.399915 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.399916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.399917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.399986 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.400300 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.400301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.400302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.400437 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:16.400517 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:16.400853 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 10:12:16.400945 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:12:16.401356 25566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.390899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:37951 (local address 127.0.0.1:51844)
0808 10:12:16.391146 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.391187 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:16.391869 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:16.392122 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:16.392125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:16.392593 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:16.392837 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.392843 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.393479 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.393480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.394212 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.394214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.394726 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.394728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.394956 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.395658 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:16.395667 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:16.397376 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:16.399517 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.399525 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.399527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.399808 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.400064 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.400065 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.400065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.400211 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.400520 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:16.400529 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:16.400690 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 10:12:16.400815 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 10:12:16.401088 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":65,"threads_started":1}
I20260808 10:12:16.401997 25561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:12:16.402751 25565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.390717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56380 (local address 127.2.30.188:35697)
0808 10:12:16.390933 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.390944 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.391124 (+ 180us) server_negotiation.cc:407] Connection header received
0808 10:12:16.391853 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.391862 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.391893 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.392006 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.392865 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.394377 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.395438 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.396003 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.396084 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.399372 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:16.399379 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:16.399380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:16.399386 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:16.401091 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.401776 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.401780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.401780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.401851 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.402094 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.402095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.402095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.402284 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:16.402450 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:16.402542 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 10:12:16.402630 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":91,"threads_started":1}
Aug 08 10:12:16 dist-test-slave-w206 krb5kdc[25320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183936, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.188@KRBTEST.COM for kudu/127.2.30.189@KRBTEST.COM
Aug 08 10:12:16 dist-test-slave-w206 krb5kdc[25320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:12:16.410550 25485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.402364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49080 (local address 127.2.30.189:36443)
0808 10:12:16.402410 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.402412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.402823 (+ 411us) server_negotiation.cc:407] Connection header received
0808 10:12:16.402896 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.402898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.402911 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.402959 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.403385 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.404189 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.405134 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.405410 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.405487 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.407635 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:16.407636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:16.407636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:16.407643 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:16.409109 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.409484 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.409485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.409486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.409552 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.409834 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.409835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.409836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.410044 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:16.410129 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:16.410376 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 10:12:16.410475 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:12:16.410588 25566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.402309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:36443 (local address 127.0.0.1:49080)
0808 10:12:16.402759 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.402772 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:16.402831 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:16.402970 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:16.403002 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:16.403224 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:16.403281 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.403283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.404293 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.404294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.405046 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.405049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.405489 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.405490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.405545 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.405994 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:16.405998 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:16.407544 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:16.409183 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.409184 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.409185 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.409405 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.409632 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.409633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.409633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.409734 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.410151 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:16.410152 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:16.410211 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 10:12:16.410313 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 10:12:16.410434 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420}
I20260808 10:12:16.411362 25561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:12:16.412587 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.403662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51858 (local address 127.2.30.190:37951)
0808 10:12:16.403706 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.403708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.403771 (+ 63us) server_negotiation.cc:407] Connection header received
0808 10:12:16.403843 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.403843 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.403851 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.403892 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.404528 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.405210 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.406208 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.406562 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.406627 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.409066 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:16.409067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:16.409068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:16.409076 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:16.411089 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.411644 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.411645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.411646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.411718 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.411987 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.411988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.411988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.412161 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:16.412365 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:16.412444 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 10:12:16.412519 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 10:12:16 dist-test-slave-w206 krb5kdc[25320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183936, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.188@KRBTEST.COM for kudu/127.2.30.188@KRBTEST.COM
Aug 08 10:12:16 dist-test-slave-w206 krb5kdc[25320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.189@KRBTEST.COM
I20260808 10:12:16.419257 25566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.411704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:35697 (local address 127.0.0.1:56386)
0808 10:12:16.411771 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.411784 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:16.411844 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:16.411987 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:16.411988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:16.412171 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:16.412273 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.412276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.413178 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.413179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.413801 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.413803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.414241 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.414242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.414294 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.414800 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:16.414808 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:16.416409 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:16.418118 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.418119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.418119 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.418290 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.418515 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.418516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.418516 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.418579 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.418857 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:16.418858 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:16.418899 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 10:12:16.419005 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 10:12:16.419122 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 10:12:16.419261 25565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.411745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56386 (local address 127.2.30.188:35697)
0808 10:12:16.411783 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.411785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.411824 (+ 39us) server_negotiation.cc:407] Connection header received
0808 10:12:16.411912 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.411912 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.411925 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.411982 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.412369 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.413079 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.413912 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.414155 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.414254 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.416518 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:16.416520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:16.416520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:16.416527 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:16.418003 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.418378 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.418379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.418379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.418444 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.418660 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.418661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.418662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.418784 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:16.418867 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:16.419057 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 10:12:16.419177 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":17}
I20260808 10:12:16.420966 25561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74: Bootstrap starting.
I20260808 10:12:16.421229 25485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.413344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49092 (local address 127.2.30.189:36443)
0808 10:12:16.413396 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.413398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.413411 (+ 13us) server_negotiation.cc:407] Connection header received
0808 10:12:16.413438 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.413439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.413450 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.413513 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.414028 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.414493 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.415287 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.415617 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.415729 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.418467 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:16.418468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:16.418468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:16.418477 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:16.420043 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.420422 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.420423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.420424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.420487 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.420696 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.420697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.420697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.420825 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:16.421013 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:16.421051 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 10:12:16.421124 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 10:12:16.422184 25561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:16.422502 25561 log.cc:824] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:16.422937 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:12:16.423399 25561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74: No bootstrap required, opened a new log
I20260808 10:12:16.424167 25376 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a0e7bdd3c06a4044a8e006e2af42769f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51858
I20260808 10:12:16.425693 25561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } }
I20260808 10:12:16.425866 25561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 722e50e9077e4f6e9eac075422818a74, State: Initialized, Role: FOLLOWER
I20260808 10:12:16.425954 25561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } }
I20260808 10:12:16.426334 25561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:16.426280 25568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } } }
I20260808 10:12:16.426460 25568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:12:16.428503 25579 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:12:16.428570 25579 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:12:16.428867 25488 master_runner.cc:186] Error getting master registration for 127.2.30.190:37951: OK, instance_id { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" instance_seqno: 1786183936026930 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:12:16.431066 25488 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:12:16.439076 25481 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } attempt: 3
Aug 08 10:12:16 dist-test-slave-w206 krb5kdc[25320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183936, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.189@KRBTEST.COM for kudu/127.2.30.188@KRBTEST.COM
I20260808 10:12:16.447333 25565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.439570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56398 (local address 127.2.30.188:35697)
0808 10:12:16.439639 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.439642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.439652 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:12:16.439680 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.439681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.439691 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.439746 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.440124 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.440609 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.441588 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.441817 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.441903 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.444249 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:16.444252 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:16.444253 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:16.444263 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:16.445850 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.446308 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.446309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.446310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.446377 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.446616 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.446617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.446617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.446845 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:16.446914 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:16.447131 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 10:12:16.447223 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:12:16.447347 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.439465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:35697 (local address 127.0.0.1:56398)
0808 10:12:16.439535 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.439549 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:16.439601 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:16.439763 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:16.439764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:16.439993 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:16.440050 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.440052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.440712 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.440713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.441517 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.441520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.441899 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.441900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.441956 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.442497 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:16.442503 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:16.444087 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:16.445945 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.445946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.445947 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.446221 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.446461 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.446462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.446462 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.446541 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.446925 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:16.446926 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:16.447015 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 10:12:16.447104 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 10:12:16.447236 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 10:12:16.449585 25481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25: Bootstrap starting.
I20260808 10:12:16.450430 25481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:16.450796 25481 log.cc:824] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:16.451839 25481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25: No bootstrap required, opened a new log
I20260808 10:12:16.454010 25481 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } }
I20260808 10:12:16.454205 25481 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1d92b5ab0d14b7aa930113184807c25, State: Initialized, Role: FOLLOWER
I20260808 10:12:16.454303 25481 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } }
I20260808 10:12:16.454701 25481 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:16.454675 25582 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } } }
I20260808 10:12:16.454778 25582 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:12:16.456900 25594 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:12:16.457041 25594 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:12:16.457051 25409 master_runner.cc:186] Error getting master registration for 127.2.30.190:37951: OK, instance_id { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" instance_seqno: 1786183936026930 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:12:16.459671 25409 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:12:16.472353 25402 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } attempt: 3
Aug 08 10:12:16 dist-test-slave-w206 krb5kdc[25320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183935, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for kudu/127.2.30.188@KRBTEST.COM
I20260808 10:12:16.481837 25404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.472760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:35697 (local address 127.0.0.1:56414)
0808 10:12:16.472823 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:16.472836 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:16.472881 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:16.473259 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:16.473261 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:16.473481 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:16.473545 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.473547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.474561 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.474562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.475474 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:16.475478 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.475977 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.475978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:16.476043 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.476585 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:16.476590 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:16.478345 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:16.480603 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.480604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.480605 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.480853 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.481092 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:16.481093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:16.481093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:16.481165 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:16.481477 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:16.481478 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:16.481533 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 10:12:16.481607 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 10:12:16.481718 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 10:12:16.481843 25565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.473058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56414 (local address 127.2.30.188:35697)
0808 10:12:16.473106 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.473108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.473119 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:12:16.473151 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.473152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.473163 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.473241 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.473685 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.474441 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.475566 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.475870 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.475961 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.478451 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:16.478453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:16.478453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:16.478463 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:16.480495 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.480939 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.480940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.480940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.481013 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.481236 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.481237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.481237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.481403 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:16.481480 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:16.481650 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 10:12:16.481742 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:12:16.484184 25402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f: Bootstrap starting.
I20260808 10:12:16.484983 25402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:16.485277 25402 log.cc:824] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:16.486227 25402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f: No bootstrap required, opened a new log
I20260808 10:12:16.487859 25402 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } }
I20260808 10:12:16.488016 25402 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e7bdd3c06a4044a8e006e2af42769f, State: Initialized, Role: FOLLOWER
I20260808 10:12:16.488169 25402 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } }
I20260808 10:12:16.488528 25402 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:16.488567 25376 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 10:12:16.488647 25376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:16.489569 25376 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } }
I20260808 10:12:16.489854 25376 leader_election.cc:288] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [CANDIDATE]: Term 1 election: Requested vote from peers c1d92b5ab0d14b7aa930113184807c25 (127.2.30.189:36443), 722e50e9077e4f6e9eac075422818a74 (127.2.30.188:35697)
I20260808 10:12:16.491127 25534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "722e50e9077e4f6e9eac075422818a74"
I20260808 10:12:16.491271 25534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:16.491554 25455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c1d92b5ab0d14b7aa930113184807c25"
I20260808 10:12:16.491716 25455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:16.492314 25534 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e7bdd3c06a4044a8e006e2af42769f in term 1.
I20260808 10:12:16.492789 25353 leader_election.cc:302] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 722e50e9077e4f6e9eac075422818a74, a0e7bdd3c06a4044a8e006e2af42769f; no voters:
I20260808 10:12:16.493017 25455 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e7bdd3c06a4044a8e006e2af42769f in term 1.
I20260808 10:12:16.493788 25615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:12:16.493952 25615 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [term 1 LEADER]: Becoming Leader. State: Replica: a0e7bdd3c06a4044a8e006e2af42769f, State: Running, Role: LEADER
I20260808 10:12:16.494036 25615 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } }
I20260808 10:12:16.494500 25615 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0e7bdd3c06a4044a8e006e2af42769f. Latest consensus state: current_term: 1 leader_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } } }
I20260808 10:12:16.494578 25615 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:16.495313 25618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:12:16.495352 25333 master_runner.cc:263] No leader master found from master a0e7bdd3c06a4044a8e006e2af42769f
I20260808 10:12:16.495407 25333 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:12:16.495572 25598 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } } }
I20260808 10:12:16.495724 25598 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:12:16.495797 25618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:12:16.497493 25534 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74 [term 1 FOLLOWER]: Refusing update from remote peer a0e7bdd3c06a4044a8e006e2af42769f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:12:16.497807 25616 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [LEADER]: Connected to new peer: Peer: permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:12:16.499251 25568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0e7bdd3c06a4044a8e006e2af42769f. Latest consensus state: current_term: 1 leader_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } } }
I20260808 10:12:16.499347 25568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:12:16.499861 25455 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25 [term 1 FOLLOWER]: Refusing update from remote peer a0e7bdd3c06a4044a8e006e2af42769f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:12:16.500160 25616 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:12:16.501348 25582 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0e7bdd3c06a4044a8e006e2af42769f. Latest consensus state: current_term: 1 leader_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } } }
I20260808 10:12:16.501449 25582 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:12:16.506449 25565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.495243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56420 (local address 127.2.30.188:35697)
0808 10:12:16.495289 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.495291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.497095 (+ 1804us) server_negotiation.cc:407] Connection header received
0808 10:12:16.497220 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.497221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.497234 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.497267 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.498451 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.499307 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.500753 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.501096 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.501141 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.503125 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:16.503126 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:16.503127 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:16.503135 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:16.505026 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.505484 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.505485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.505486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.505551 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.505776 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.505777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.505777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.505924 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:16.506001 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:16.506254 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 10:12:16.506345 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:12:16.508270 25568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } } }
I20260808 10:12:16.508358 25624 mvcc.cc:205] Tried to move back new op lower bound from 7316209403891044352 to 7316209403880411136. Current Snapshot: MvccSnapshot[applied={T|T < 7316209403891044352}]
I20260808 10:12:16.508297 25598 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } } }
I20260808 10:12:16.508380 25568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:12:16.508409 25598 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:16.508757 25622 mvcc.cc:205] Tried to move back new op lower bound from 7316209403891044352 to 7316209403880411136. Current Snapshot: MvccSnapshot[applied={T|T < 7316209403891044352}]
I20260808 10:12:16.510049 25616 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } } }
I20260808 10:12:16.510138 25616 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:16.510712 25582 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e7bdd3c06a4044a8e006e2af42769f" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 37951 } } peers { permanent_uuid: "c1d92b5ab0d14b7aa930113184807c25" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 36443 } } peers { permanent_uuid: "722e50e9077e4f6e9eac075422818a74" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 35697 } } }
I20260808 10:12:16.510813 25582 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:12:16.511243 25626 mvcc.cc:205] Tried to move back new op lower bound from 7316209403891044352 to 7316209403880411136. Current Snapshot: MvccSnapshot[applied={T|T < 7316209403891044352}]
I20260808 10:12:16.522679 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.496301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51868 (local address 127.2.30.190:37951)
0808 10:12:16.496366 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.496368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.507153 (+ 10785us) server_negotiation.cc:407] Connection header received
0808 10:12:16.507267 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.507269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.507282 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.507314 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.509096 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.509880 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.512453 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.512745 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.512791 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.517787 (+ 4996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:16.517789 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:16.517789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:16.517799 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:16.519832 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.521616 (+ 1784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.521618 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.521618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.521688 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.521936 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.521937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.521937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.522096 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:16.522222 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:16.522479 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 10:12:16.522570 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:12:16.528028 25485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:16.496740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49094 (local address 127.2.30.189:36443)
0808 10:12:16.496783 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.496785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.507185 (+ 10400us) server_negotiation.cc:407] Connection header received
0808 10:12:16.507310 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.507312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.507326 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.507358 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.513585 (+ 6227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.514336 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.516824 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:16.517156 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:16.517222 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:16.523782 (+ 6560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:16.523784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:16.523784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:16.523794 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:16.525955 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.526503 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.526504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.526504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.526571 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:16.527238 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:16.527240 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:16.527240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:16.527419 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:16.527531 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:16.527819 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 10:12:16.527914 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:12:16.529981 25618 catalog_manager.cc:1347] Generated new cluster ID: 86e62a20407f443ab854853c3fd5d574
I20260808 10:12:16.530053 25618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:12:16.539654 25618 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:12:16.540505 25618 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:12:16.546725 25618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0e7bdd3c06a4044a8e006e2af42769f: Generated new TSK 0
I20260808 10:12:16.546931 25618 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:12:16.572702 25565 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:16.570649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56430 (local address 127.2.30.188:35697)
0808 10:12:16.570699 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.570701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.571780 (+ 1079us) server_negotiation.cc:407] Connection header received
0808 10:12:16.571919 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.571920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.571933 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.571965 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.572618 (+ 653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56430: BlockingRecv error: recv got EOF from 127.0.0.1:56430 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 10:12:16.573668 25405 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:16.571119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51882 (local address 127.2.30.190:37951)
0808 10:12:16.571163 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.571165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.573007 (+ 1842us) server_negotiation.cc:407] Connection header received
0808 10:12:16.573132 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.573134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.573147 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.573179 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.573599 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51882: BlockingRecv error: recv got EOF from 127.0.0.1:51882 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 10:12:16.574487 25485 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:16.571490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49098 (local address 127.2.30.189:36443)
0808 10:12:16.571532 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.571534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.573905 (+ 2371us) server_negotiation.cc:407] Connection header received
0808 10:12:16.574007 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.574009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.574020 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.574050 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.574420 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49098: BlockingRecv error: recv got EOF from 127.0.0.1:49098 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 10:12:16.577280 25485 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:16.576434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49114 (local address 127.2.30.189:36443)
0808 10:12:16.576474 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.576475 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.576693 (+ 218us) server_negotiation.cc:407] Connection header received
0808 10:12:16.576802 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.576803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.576835 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.576868 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.577228 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49114: BlockingRecv error: recv got EOF from 127.0.0.1:49114 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 10:12:16.580559 25485 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:16.579683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49116 (local address 127.2.30.189:36443)
0808 10:12:16.579721 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.579722 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.580012 (+ 290us) server_negotiation.cc:407] Connection header received
0808 10:12:16.580116 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.580117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.580127 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.580157 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.580504 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49116: BlockingRecv error: recv got EOF from 127.0.0.1:49116 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 10:12:16.584664 25485 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:16.583838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49132 (local address 127.2.30.189:36443)
0808 10:12:16.583880 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.583882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.584129 (+ 247us) server_negotiation.cc:407] Connection header received
0808 10:12:16.584228 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.584229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.584239 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.584269 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.584613 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49132: BlockingRecv error: recv got EOF from 127.0.0.1:49132 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 10:12:16.591115 25485 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:16.590113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49136 (local address 127.2.30.189:36443)
0808 10:12:16.590173 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.590175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.590457 (+ 282us) server_negotiation.cc:407] Connection header received
0808 10:12:16.590563 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.590564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.590575 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.590606 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.591054 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49136: BlockingRecv error: recv got EOF from 127.0.0.1:49136 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 10:12:16.601634 25485 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:16.600625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49142 (local address 127.2.30.189:36443)
0808 10:12:16.600677 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.600680 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.600990 (+ 310us) server_negotiation.cc:407] Connection header received
0808 10:12:16.601103 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.601104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.601117 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.601149 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.601571 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49142: BlockingRecv error: recv got EOF from 127.0.0.1:49142 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 10:12:16.620203 25485 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:16.619162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49158 (local address 127.2.30.189:36443)
0808 10:12:16.619212 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.619214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.619518 (+ 304us) server_negotiation.cc:407] Connection header received
0808 10:12:16.619629 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.619630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.619643 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.619674 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.620140 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49158: BlockingRecv error: recv got EOF from 127.0.0.1:49158 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 10:12:16.654537 25485 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:16.653748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49172 (local address 127.2.30.189:36443)
0808 10:12:16.653834 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.653836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.653969 (+ 133us) server_negotiation.cc:407] Connection header received
0808 10:12:16.654010 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.654011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.654026 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.654104 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.654465 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49172: BlockingRecv error: recv got EOF from 127.0.0.1:49172 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 10:12:16.720722 25485 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:16.719821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49182 (local address 127.2.30.189:36443)
0808 10:12:16.720033 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.720035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.720102 (+ 67us) server_negotiation.cc:407] Connection header received
0808 10:12:16.720148 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.720149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.720161 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.720217 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.720633 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49182: BlockingRecv error: recv got EOF from 127.0.0.1:49182 (error 108)
Metrics: {"server-negotiator.queue_time_us":171}
W20260808 10:12:16.850895 25485 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:16.850052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49194 (local address 127.2.30.189:36443)
0808 10:12:16.850173 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:16.850175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:16.850249 (+ 74us) server_negotiation.cc:407] Connection header received
0808 10:12:16.850331 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:16.850332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:16.850342 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:16.850387 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:16.850832 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49194: BlockingRecv error: recv got EOF from 127.0.0.1:49194 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260808 10:12:17.108871 25485 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:17.108262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49210 (local address 127.2.30.189:36443)
0808 10:12:17.108382 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:17.108384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:17.108394 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:12:17.108427 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:17.108428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:17.108438 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:17.108492 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:17.108815 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49210: BlockingRecv error: recv got EOF from 127.0.0.1:49210 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 10:12:17.429049 25579 catalog_manager.cc:1259] Loaded cluster ID: 86e62a20407f443ab854853c3fd5d574
I20260808 10:12:17.429131 25579 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74: loading cluster ID for follower catalog manager: success
I20260808 10:12:17.429965 25579 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74: acquiring CA information for follower catalog manager: success
I20260808 10:12:17.430294 25579 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 722e50e9077e4f6e9eac075422818a74: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:12:17.457587 25594 catalog_manager.cc:1259] Loaded cluster ID: 86e62a20407f443ab854853c3fd5d574
I20260808 10:12:17.457677 25594 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25: loading cluster ID for follower catalog manager: success
I20260808 10:12:17.458587 25594 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25: acquiring CA information for follower catalog manager: success
I20260808 10:12:17.458946 25594 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c1d92b5ab0d14b7aa930113184807c25: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:12:17.624930 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:17.622166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49216 (local address 127.2.30.189:36443)
0808 10:12:17.622309 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:17.622311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:17.622321 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:12:17.622374 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:17.622375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:17.622455 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:17.622546 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:12:17.622712 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:17.623294 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:17.624064 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:17.624289 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:17.624371 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:17.624555 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:12:17.624665 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:12:17.624751 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:17.624794 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 10:12:17.624816 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
I20260808 10:12:17.630087 25703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:17.627279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56440 (local address 127.2.30.188:35697)
0808 10:12:17.627427 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:17.627429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:17.627516 (+ 87us) server_negotiation.cc:407] Connection header received
0808 10:12:17.627650 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:17.627651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:17.627712 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:17.627804 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:12:17.627964 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:17.628463 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:17.629224 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:17.629447 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:17.629531 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:17.629708 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:12:17.629833 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:12:17.629924 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:17.629964 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 10:12:17.629988 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":54,"threads_started":1}
I20260808 10:12:17.634629 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:17.631914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49226 (local address 127.2.30.189:36443)
0808 10:12:17.631994 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:17.631997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:17.632043 (+ 46us) server_negotiation.cc:407] Connection header received
0808 10:12:17.632114 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:17.632116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:17.632129 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:17.632179 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:12:17.632267 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:17.632783 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:17.633838 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:17.634137 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:17.634209 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:17.634358 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:12:17.634393 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:12:17.634444 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:17.634508 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 10:12:17.634534 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 10:12:17.635442 25703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:17.631986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56456 (local address 127.2.30.188:35697)
0808 10:12:17.632025 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:17.632027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:17.632725 (+ 698us) server_negotiation.cc:407] Connection header received
0808 10:12:17.632813 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:17.632814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:17.632825 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:17.632868 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:12:17.633030 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:17.633493 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:17.634602 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:17.634887 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:17.634930 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:17.635149 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:12:17.635181 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:12:17.635254 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:17.635336 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 10:12:17.635357 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 10:12:17.635514 25710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:17.631910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51884 (local address 127.2.30.190:37951)
0808 10:12:17.632170 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:17.632172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:17.632408 (+ 236us) server_negotiation.cc:407] Connection header received
0808 10:12:17.632450 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:17.632451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:17.632490 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:17.632595 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:12:17.632760 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:17.633529 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:17.634407 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:17.634862 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:17.634919 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:17.635173 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:12:17.635266 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:12:17.635344 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:17.635402 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 10:12:17.635425 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":59,"threads_started":1}
I20260808 10:12:17.637351 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 25333
I20260808 10:12:17.644476 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 25409
I20260808 10:12:17.651324 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 25488
2026-08-08T10:12:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2978 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 10:12:17 dist-test-slave-w206 krb5kdc[25715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 10:12:17 dist-test-slave-w206 krb5kdc[25715](info): set up 2 sockets
Aug 08 10:12:17 dist-test-slave-w206 krb5kdc[25715](info): commencing operation
krb5kdc: starting...
W20260808 10:12:18.748169 2170 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/test-user.keytab.
Aug 08 10:12:18 dist-test-slave-w206 krb5kdc[25715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T10:12:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T10:12:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.30.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
Entry for principal kudu/127.2.30.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab.
I20260808 10:12:18.882879 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:41027
--webserver_interface=127.2.30.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:37239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:41027,127.2.30.189:33359,127.2.30.188:34277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 10:12:18.970180 25728 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:12:18.970444 25728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:18.970463 25728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:18.970485 25728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:18.971689 25728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:18.971732 25728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:18.971746 25728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:18.971760 25728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:18.971772 25728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:18.973429 25728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:37239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:41027,127.2.30.189:33359,127.2.30.188:34277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.190.keytab
--principal=kudu/127.2.30.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.190:41027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.30.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:18.973675 25728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:18.973875 25728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:12:18.979318 25733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:18.979360 25736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:18.979318 25734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:18.979566 25728 server_base.cc:1034] running on GCE node
I20260808 10:12:18.979933 25728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:18.980285 25728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:18.981446 25728 hybrid_clock.cc:648] HybridClock initialized: now 1786183938981420 us; error 41 us; skew 500 ppm
Aug 08 10:12:18 dist-test-slave-w206 krb5kdc[25715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183938, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:18.987969 25728 init.cc:377] Logged in from keytab as kudu/127.2.30.190@KRBTEST.COM (short username kudu)
I20260808 10:12:18.988478 25728 webserver.cc:460] Webserver started at http://127.2.30.190:37569/ using document root <none> and password file <none>
I20260808 10:12:18.988737 25728 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:18.988807 25728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:18.988966 25728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:18.990074 25728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/data/instance:
uuid: "8e1b7670c9ec4e089203c4284a16e7a6"
format_stamp: "Formatted at 2026-08-08 10:12:18 on dist-test-slave-w206"
server_key: "68257a54e763fe5eb14c2ad00eabc66b"
server_key_iv: "28c0075163b397c39039985d91fddcaf"
server_key_version: "encryptionkey@0"
I20260808 10:12:18.990439 25728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/wal/instance:
uuid: "8e1b7670c9ec4e089203c4284a16e7a6"
format_stamp: "Formatted at 2026-08-08 10:12:18 on dist-test-slave-w206"
server_key: "68257a54e763fe5eb14c2ad00eabc66b"
server_key_iv: "28c0075163b397c39039985d91fddcaf"
server_key_version: "encryptionkey@0"
I20260808 10:12:18.992026 25728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 10:12:18.992831 25743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:18.993024 25728 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 10:12:18.993139 25728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/wal
uuid: "8e1b7670c9ec4e089203c4284a16e7a6"
format_stamp: "Formatted at 2026-08-08 10:12:18 on dist-test-slave-w206"
server_key: "68257a54e763fe5eb14c2ad00eabc66b"
server_key_iv: "28c0075163b397c39039985d91fddcaf"
server_key_version: "encryptionkey@0"
I20260808 10:12:18.993217 25728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:19.016979 25728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:19.019137 25728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:19.019309 25728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:19.023934 25728 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.190:41027
I20260808 10:12:19.023998 25800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.190:41027 every 8 connection(s)
I20260808 10:12:19.024394 25728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/data/info.pb
I20260808 10:12:19.025455 25801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:19.027151 25801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:12:19.029757 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 25728
I20260808 10:12:19.029872 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-0/wal/instance
I20260808 10:12:19.030066 2170 external_mini_cluster.cc:1428] Setting key 420f507ecd49d4749b6600fa2481ec41
Aug 08 10:12:19 dist-test-slave-w206 krb5kdc[25715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183938, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:12:19.038785 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.027918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41027 (local address 127.0.0.1:47150)
0808 10:12:19.028117 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.028158 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:19.028628 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:19.028905 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:19.028907 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:19.029192 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:19.029364 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.029368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.030558 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.030560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.031757 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.031760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.032439 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.032440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.032511 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.033221 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:19.033230 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:19.034828 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:19.036846 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.036859 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.036862 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.037067 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.037406 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.037406 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.037407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.037533 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.038060 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:19.038093 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:19.038191 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 10:12:19.038317 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 10:12:19.038578 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":61,"thread_start_us":109,"threads_started":1}
I20260808 10:12:19.038785 25804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.027936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47150 (local address 127.2.30.190:41027)
0808 10:12:19.028136 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.028146 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.028261 (+ 115us) server_negotiation.cc:407] Connection header received
0808 10:12:19.028763 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.028765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.028783 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.028874 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.029463 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.030457 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.031869 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.032213 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.032469 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.034929 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:19.034938 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:19.034938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:19.034948 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:19.036735 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.037170 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.037174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.037176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.037294 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.037605 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.037606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.037606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.037995 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:19.038086 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:19.038337 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 10:12:19.038501 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for kudu/127.2.30.189@KRBTEST.COM; defaulting to no policy
I20260808 10:12:19.039660 25801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:12:19.040069 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.039932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:33359 (local address 127.0.0.1:58920)
0808 10:12:19.039988 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.040044 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:33359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 10:12:19.040283 25746 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.30.189:33359: connect: Connection refused (error 111)
I20260808 10:12:19.040804 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.040702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:33359 (local address 127.0.0.1:58924)
0808 10:12:19.040761 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.040788 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:33359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 10:12:19.040954 25801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.189:33359: Network error: Client connection negotiation failed: client connection to 127.2.30.189:33359: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.189.keytab.
Entry for principal kudu/127.2.30.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.189.keytab.
I20260808 10:12:19.062208 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.189:33359
--webserver_interface=127.2.30.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.189.keytab
--principal=kudu/127.2.30.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:37239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:41027,127.2.30.189:33359,127.2.30.188:34277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:12:19.090189 25801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } attempt: 1
I20260808 10:12:19.090801 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.090608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:33359 (local address 127.0.0.1:58934)
0808 10:12:19.090697 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.090755 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:33359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 10:12:19.091625 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.091507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:33359 (local address 127.0.0.1:58938)
0808 10:12:19.091551 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.091590 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:33359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 10:12:19.091810 25801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.189:33359: Network error: Client connection negotiation failed: client connection to 127.2.30.189:33359: connect: Connection refused (error 111)
W20260808 10:12:19.157970 25809 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:12:19.158296 25809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:19.158330 25809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:19.158378 25809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:19.159790 25809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:19.159873 25809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:19.159922 25809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:19.159956 25809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:19.159989 25809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:19.160066 25801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } attempt: 2
I20260808 10:12:19.160730 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.160551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:33359 (local address 127.0.0.1:58946)
0808 10:12:19.160633 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.160686 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:33359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 10:12:19.161172 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.161061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:33359 (local address 127.0.0.1:58958)
0808 10:12:19.161111 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.161151 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.189:33359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 10:12:19.161271 25801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.189:33359: Network error: Client connection negotiation failed: client connection to 127.2.30.189:33359: connect: Connection refused (error 111)
I20260808 10:12:19.162024 25809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:37239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:41027,127.2.30.189:33359,127.2.30.188:34277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.189.keytab
--principal=kudu/127.2.30.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.189:33359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.30.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:19.162370 25809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:19.162633 25809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:12:19.168043 25815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:19.168001 25817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:19.168010 25814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:19.168460 25809 server_base.cc:1034] running on GCE node
I20260808 10:12:19.168766 25809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:19.169124 25809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:19.170302 25809 hybrid_clock.cc:648] HybridClock initialized: now 1786183939170288 us; error 47 us; skew 500 ppm
Aug 08 10:12:19 dist-test-slave-w206 krb5kdc[25715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183939, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:19.173396 25809 init.cc:377] Logged in from keytab as kudu/127.2.30.189@KRBTEST.COM (short username kudu)
I20260808 10:12:19.173969 25809 webserver.cc:460] Webserver started at http://127.2.30.189:42661/ using document root <none> and password file <none>
I20260808 10:12:19.174212 25809 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:19.174283 25809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:19.174458 25809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:19.175803 25809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/data/instance:
uuid: "23a8b6b318874db89f490656f73f2476"
format_stamp: "Formatted at 2026-08-08 10:12:19 on dist-test-slave-w206"
server_key: "d72bffde211f39c3859f2c7dc6bc91bd"
server_key_iv: "4067ac61ed748ec4a15db46fcaa75181"
server_key_version: "encryptionkey@0"
I20260808 10:12:19.176203 25809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/wal/instance:
uuid: "23a8b6b318874db89f490656f73f2476"
format_stamp: "Formatted at 2026-08-08 10:12:19 on dist-test-slave-w206"
server_key: "d72bffde211f39c3859f2c7dc6bc91bd"
server_key_iv: "4067ac61ed748ec4a15db46fcaa75181"
server_key_version: "encryptionkey@0"
I20260808 10:12:19.177714 25809 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 10:12:19.178658 25824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:19.178871 25809 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:12:19.179006 25809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/wal
uuid: "23a8b6b318874db89f490656f73f2476"
format_stamp: "Formatted at 2026-08-08 10:12:19 on dist-test-slave-w206"
server_key: "d72bffde211f39c3859f2c7dc6bc91bd"
server_key_iv: "4067ac61ed748ec4a15db46fcaa75181"
server_key_version: "encryptionkey@0"
I20260808 10:12:19.179103 25809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:19.210605 25809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:19.213102 25809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:19.213325 25809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:19.217909 25809 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.189:33359
I20260808 10:12:19.217989 25878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.189:33359 every 8 connection(s)
I20260808 10:12:19.218400 25809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/data/info.pb
I20260808 10:12:19.219488 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 25809
I20260808 10:12:19.219601 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-1/wal/instance
I20260808 10:12:19.219820 2170 external_mini_cluster.cc:1428] Setting key fd01d5f40b3513e9afb50657ec96bb97
I20260808 10:12:19.219818 25879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:19.221597 25879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.30.188@KRBTEST.COM; defaulting to no policy
Aug 08 10:12:19 dist-test-slave-w206 krb5kdc[25715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183939, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.189@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.30.188@KRBTEST.COM" created.
I20260808 10:12:19.232118 25804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.222550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47154 (local address 127.2.30.190:41027)
0808 10:12:19.222618 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.222621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.222859 (+ 238us) server_negotiation.cc:407] Connection header received
0808 10:12:19.223582 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.223584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.223601 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.223660 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.224503 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.225019 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.226093 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.226311 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.226394 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.228813 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:19.228815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:19.228815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:19.228824 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:19.230299 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.230762 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.230762 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.230763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.230818 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.231261 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.231262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.231262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.231430 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:19.231511 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:19.231957 (+ 446us) server_negotiation.cc:299] Negotiation successful
0808 10:12:19.232038 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 10:12:19.232527 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.222469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41027 (local address 127.0.0.1:47154)
0808 10:12:19.222715 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.222759 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:19.223453 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:19.223733 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:19.223737 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:19.224029 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:19.224399 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.224404 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.225131 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.225132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.225976 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.225979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.226417 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.226419 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.226571 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.227220 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:19.227229 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:19.228708 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:19.230410 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.230419 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.230422 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.230684 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.230882 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.230883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.230883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.231110 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.231563 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:19.231574 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:19.231780 (+ 206us) client_negotiation.cc:769] Sending connection context
0808 10:12:19.231903 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 10:12:19.232101 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":110,"threads_started":1}
I20260808 10:12:19.233256 25879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:12:19 dist-test-slave-w206 krb5kdc[25715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183939, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.189@KRBTEST.COM for kudu/127.2.30.189@KRBTEST.COM
I20260808 10:12:19.241331 25885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.233663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58974 (local address 127.2.30.189:33359)
0808 10:12:19.233816 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.233820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.233843 (+ 23us) server_negotiation.cc:407] Connection header received
0808 10:12:19.233862 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.233864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.233875 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.233965 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.234266 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.235033 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.235736 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.236149 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.236241 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.238144 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:19.238156 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:19.238156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:19.238164 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:19.239943 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.240338 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.240341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.240342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.240406 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.240605 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.240606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.240606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.240859 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:19.240963 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:19.241074 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 10:12:19.241173 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":97,"threads_started":1}
I20260808 10:12:19.241334 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.233579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:33359 (local address 127.0.0.1:58974)
0808 10:12:19.233630 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.233641 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:19.233679 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:19.233965 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:19.233966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:19.234121 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:19.234177 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.234178 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.235125 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.235126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.235637 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.235638 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.236241 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.236242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.236309 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.236791 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:19.236796 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:19.238052 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:19.240058 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.240059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.240059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.240235 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.240475 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.240476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.240476 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.240533 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.240936 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:19.240937 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:19.240979 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 10:12:19.241045 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 10:12:19.241239 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":96}
I20260808 10:12:19.242102 25879 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:12:19.242560 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.242420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:34277 (local address 127.0.0.1:59840)
0808 10:12:19.242475 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.242529 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 10:12:19.242738 25826 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
I20260808 10:12:19.243691 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.243545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:34277 (local address 127.0.0.1:59852)
0808 10:12:19.243627 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.243661 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 10:12:19.243939 25879 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:34277: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.30.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.188.keytab.
Entry for principal kudu/127.2.30.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.188.keytab.
I20260808 10:12:19.251410 2170 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskbFyXgX/build/release/bin/kudu
/tmp/dist-test-taskbFyXgX/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.188:34277
--webserver_interface=127.2.30.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.188.keytab
--principal=kudu/127.2.30.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.30.148:37239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.30.190:41027,127.2.30.189:33359,127.2.30.188:34277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 10:12:19.252417 25801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } attempt: 3
Aug 08 10:12:19 dist-test-slave-w206 krb5kdc[25715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183938, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for kudu/127.2.30.189@KRBTEST.COM
I20260808 10:12:19.259791 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.252789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:33359 (local address 127.0.0.1:58982)
0808 10:12:19.252839 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.252851 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:19.252923 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:19.253085 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:19.253086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:19.253252 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:19.253322 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.253324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.254026 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.254028 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.254561 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.254562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.255046 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.255047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.255116 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.255658 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:19.255663 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:19.256917 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:19.258655 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.258656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.258656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.258836 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.259097 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.259098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.259098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.259164 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.259436 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:19.259437 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:19.259489 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 10:12:19.259571 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 10:12:19.259686 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 10:12:19.259791 25885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.252871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58982 (local address 127.2.30.189:33359)
0808 10:12:19.252944 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.252946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.252968 (+ 22us) server_negotiation.cc:407] Connection header received
0808 10:12:19.253003 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.253004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.253013 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.253078 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.253430 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.253937 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.254652 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.254891 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.254970 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.257022 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:19.257023 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:19.257024 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:19.257039 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:19.258559 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.258912 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.258913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.258914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.259018 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.259229 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.259230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.259231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.259357 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:19.259443 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:19.259598 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 10:12:19.259687 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 10:12:19.260177 25801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } has no permanent_uuid. Determining permanent_uuid...
I20260808 10:12:19.260555 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.260446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:34277 (local address 127.0.0.1:59856)
0808 10:12:19.260499 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.260528 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 10:12:19.260999 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.260910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:34277 (local address 127.0.0.1:59868)
0808 10:12:19.260959 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.260983 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 10:12:19.261116 25801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:34277: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
I20260808 10:12:19.292348 25801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } attempt: 1
I20260808 10:12:19.292944 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.292783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:34277 (local address 127.0.0.1:59878)
0808 10:12:19.292863 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.292910 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 10:12:19.293215 25879 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } attempt: 1
I20260808 10:12:19.293385 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.293233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:34277 (local address 127.0.0.1:59888)
0808 10:12:19.293300 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.293337 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 10:12:19.293488 25801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:34277: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
I20260808 10:12:19.293655 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.293518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:34277 (local address 127.0.0.1:59890)
0808 10:12:19.293585 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.293620 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 10:12:19.294065 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.293971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:34277 (local address 127.0.0.1:59894)
0808 10:12:19.294020 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.294048 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 10:12:19.294202 25879 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:34277: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
W20260808 10:12:19.347155 25887 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 10:12:19.347482 25887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 10:12:19.347525 25887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 10:12:19.347589 25887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 10:12:19.348922 25887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 10:12:19.349013 25887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 10:12:19.349038 25887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 10:12:19.349061 25887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 10:12:19.349092 25887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 10:12:19.350802 25887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.30.148:37239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.30.190:41027,127.2.30.189:33359,127.2.30.188:34277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/krb5kdc/kudu_127.2.30.188.keytab
--principal=kudu/127.2.30.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.30.188:34277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.30.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 08:05:23 UTC on ccycloud.quasar-slepfe.root.comops.site
build id 4574261
I20260808 10:12:19.351162 25887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 10:12:19.351435 25887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 10:12:19.357254 25893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:19.357246 25895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 10:12:19.357223 25892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 10:12:19.357645 25887 server_base.cc:1034] running on GCE node
I20260808 10:12:19.357925 25887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 10:12:19.358315 25887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 10:12:19.359508 25887 hybrid_clock.cc:648] HybridClock initialized: now 1786183939359486 us; error 55 us; skew 500 ppm
Aug 08 10:12:19 dist-test-slave-w206 krb5kdc[25715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183939, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 10:12:19.362366 25879 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } attempt: 2
I20260808 10:12:19.362619 25887 init.cc:377] Logged in from keytab as kudu/127.2.30.188@KRBTEST.COM (short username kudu)
I20260808 10:12:19.362782 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.362642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:34277 (local address 127.0.0.1:59902)
0808 10:12:19.362704 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.362739 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 10:12:19.363178 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.363076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:34277 (local address 127.0.0.1:59908)
0808 10:12:19.363127 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.363153 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 10:12:19.363276 25879 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:34277: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
I20260808 10:12:19.363329 25887 webserver.cc:460] Webserver started at http://127.2.30.188:38605/ using document root <none> and password file <none>
I20260808 10:12:19.363596 25887 fs_manager.cc:359] Metadata directory not provided
I20260808 10:12:19.363653 25887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 10:12:19.363834 25887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 10:12:19.365073 25887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/data/instance:
uuid: "d63b66333b164e069820aa416001d92e"
format_stamp: "Formatted at 2026-08-08 10:12:19 on dist-test-slave-w206"
server_key: "113b8472908617e8054adfed54edfb0b"
server_key_iv: "476de23da7fcc77fd22bcd31634b43aa"
server_key_version: "encryptionkey@0"
I20260808 10:12:19.365522 25887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/wal/instance:
uuid: "d63b66333b164e069820aa416001d92e"
format_stamp: "Formatted at 2026-08-08 10:12:19 on dist-test-slave-w206"
server_key: "113b8472908617e8054adfed54edfb0b"
server_key_iv: "476de23da7fcc77fd22bcd31634b43aa"
server_key_version: "encryptionkey@0"
I20260808 10:12:19.367129 25887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 10:12:19.368069 25902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 10:12:19.368355 25887 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 10:12:19.368463 25887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/data,/tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/wal
uuid: "d63b66333b164e069820aa416001d92e"
format_stamp: "Formatted at 2026-08-08 10:12:19 on dist-test-slave-w206"
server_key: "113b8472908617e8054adfed54edfb0b"
server_key_iv: "476de23da7fcc77fd22bcd31634b43aa"
server_key_version: "encryptionkey@0"
I20260808 10:12:19.368561 25887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 10:12:19.368641 25801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } attempt: 2
I20260808 10:12:19.369065 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.368940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:34277 (local address 127.0.0.1:59910)
0808 10:12:19.368999 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.369039 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 10:12:19.369419 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.369313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:34277 (local address 127.0.0.1:59926)
0808 10:12:19.369367 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.369392 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 10:12:19.369510 25801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.30.188:34277: Network error: Client connection negotiation failed: client connection to 127.2.30.188:34277: connect: Connection refused (error 111)
I20260808 10:12:19.379063 25887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 10:12:19.381906 25887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 10:12:19.382128 25887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 10:12:19.387068 25887 rpc_server.cc:307] RPC server started. Bound to: 127.2.30.188:34277
I20260808 10:12:19.387109 25958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.30.188:34277 every 8 connection(s)
I20260808 10:12:19.387552 25887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/data/info.pb
I20260808 10:12:19.388449 2170 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskbFyXgX/build/release/bin/kudu as pid 25887
I20260808 10:12:19.388576 2170 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskbFyXgX/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786183651681294-2170-0/minicluster-data/master-2/wal/instance
I20260808 10:12:19.388801 2170 external_mini_cluster.cc:1428] Setting key 3b11ae58baac3dc22f60f5c77ec7d121
I20260808 10:12:19.388811 25959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 10:12:19.391577 25959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:12:19 dist-test-slave-w206 krb5kdc[25715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.188@KRBTEST.COM
Aug 08 10:12:19 dist-test-slave-w206 krb5kdc[25715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183939, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.188@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
I20260808 10:12:19.400410 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.389443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59928 (local address 127.2.30.188:34277)
0808 10:12:19.389658 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.389666 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.389701 (+ 35us) server_negotiation.cc:407] Connection header received
0808 10:12:19.390129 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.390136 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.390187 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.390316 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.391237 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.392377 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.393207 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.393694 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.393779 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.396761 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:19.396771 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:19.396771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:19.396780 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:19.398517 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.398970 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.399005 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.399006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.399129 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.399361 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.399362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.399362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.399626 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:19.399737 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:19.399960 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 10:12:19.400128 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":102,"threads_started":1}
I20260808 10:12:19.401343 25804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.392435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47160 (local address 127.2.30.190:41027)
0808 10:12:19.392484 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.392486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.392644 (+ 158us) server_negotiation.cc:407] Connection header received
0808 10:12:19.392730 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.392732 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.392744 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.392823 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.393402 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.393951 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.394878 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.395264 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.395337 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.398023 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:19.398024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:19.398025 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:19.398032 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:19.399740 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.400282 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.400283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.400283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.400339 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.400727 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.400728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.400728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.400848 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:19.400931 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:19.401143 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 10:12:19.401244 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 10:12:19.401609 25964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.392261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.190:41027 (local address 127.0.0.1:47160)
0808 10:12:19.392541 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.392576 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:19.392665 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:19.392847 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:19.392850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:19.393151 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:19.393269 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.393271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.394072 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.394073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.394778 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.394781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.395391 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.395392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.395448 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.395997 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:19.396005 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:19.397913 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:19.399815 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.399817 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.399819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.400159 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.400410 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.400411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.400411 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.400509 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.400924 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:19.400931 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:19.400990 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 10:12:19.401101 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 10:12:19.401255 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":137,"threads_started":1}
I20260808 10:12:19.402115 25959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:12:19 dist-test-slave-w206 krb5kdc[25715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.190@KRBTEST.COM
Aug 08 10:12:19 dist-test-slave-w206 krb5kdc[25715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183939, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.188@KRBTEST.COM for kudu/127.2.30.189@KRBTEST.COM
I20260808 10:12:19.410380 25804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.401388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47164 (local address 127.2.30.190:41027)
0808 10:12:19.401418 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.401420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.401430 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:12:19.401482 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.401483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.401495 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.401538 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.402153 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.402890 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.403942 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.404225 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.404323 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.406935 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:19.406936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:19.406937 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:19.406944 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:19.408928 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.409405 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.409406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.409406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.409463 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.409683 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.409684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.409684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.409844 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:19.410073 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:19.410115 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 10:12:19.410204 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14,"thread_start_us":129,"threads_started":1}
I20260808 10:12:19.410560 25964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.402414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.189:33359 (local address 127.0.0.1:58992)
0808 10:12:19.402457 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.402466 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:19.402498 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:19.402654 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:19.402655 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:19.402803 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:19.402855 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.402857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.403942 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.403944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.404722 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.404726 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.405262 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.405263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.405316 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.405762 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:19.405767 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:19.407351 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:19.409323 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.409324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.409324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.409575 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.409814 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.409815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.409815 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.409881 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.410200 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:19.410201 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:19.410253 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 10:12:19.410329 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 10:12:19.410452 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 10:12:19.410557 25885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.402493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58992 (local address 127.2.30.189:33359)
0808 10:12:19.402537 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.402539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.402547 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:12:19.402571 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.402572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.402588 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.402633 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.403011 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.403812 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.404831 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.405171 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.405253 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.407477 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:19.407479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:19.407479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:19.407487 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:19.409179 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.409671 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.409673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.409674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.409738 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.409969 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.409970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.409970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.410114 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:19.410181 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:19.410384 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 10:12:19.410465 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 10:12:19.411439 25959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 10:12:19 dist-test-slave-w206 krb5kdc[25715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183939, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.188@KRBTEST.COM for kudu/127.2.30.188@KRBTEST.COM
Aug 08 10:12:19 dist-test-slave-w206 krb5kdc[25715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.30.189@KRBTEST.COM
I20260808 10:12:19.418484 25964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.411749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:34277 (local address 127.0.0.1:59940)
0808 10:12:19.411779 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.411789 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:19.411828 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:19.411965 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:19.411965 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:19.412117 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:19.412152 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.412154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.412831 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.412832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.413440 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.413442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.413837 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.413838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.413884 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.414412 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:19.414417 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:19.415712 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:19.417391 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.417392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.417393 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.417559 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.417775 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.417776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.417776 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.417833 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.418091 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:19.418092 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:19.418132 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 10:12:19.418243 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 10:12:19.418363 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260808 10:12:19.418484 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.411783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59940 (local address 127.2.30.188:34277)
0808 10:12:19.411847 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.411849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.411859 (+ 10us) server_negotiation.cc:407] Connection header received
0808 10:12:19.411881 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.411882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.411892 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.411964 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.412230 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.412741 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.413522 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.413760 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.413830 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.415802 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:19.415803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:19.415803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:19.415811 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:19.417254 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.417647 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.417648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.417649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.417702 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.417901 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.417902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.417902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.418025 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:19.418100 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:19.418265 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 10:12:19.418405 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":40}
I20260808 10:12:19.419339 25885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.411340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59006 (local address 127.2.30.189:33359)
0808 10:12:19.411377 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.411379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.411388 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:12:19.411407 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.411408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.411418 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.411468 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.412081 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.412617 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.413334 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.413631 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.413684 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.416481 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:19.416483 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:19.416483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:19.416491 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:19.417970 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.418334 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.418335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.418336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.418396 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.418631 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.418632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.418633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.418794 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:19.418959 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:19.419163 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 10:12:19.419243 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 10:12:19.420495 25959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e: Bootstrap starting.
I20260808 10:12:19.421010 2170 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 10:12:19.421597 25959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:19.421974 25959 log.cc:824] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:19.422793 25959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e: No bootstrap required, opened a new log
I20260808 10:12:19.422317 25773 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8e1b7670c9ec4e089203c4284a16e7a6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47164
I20260808 10:12:19.424597 25959 raft_consensus.cc:348] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } }
I20260808 10:12:19.424953 25959 raft_consensus.cc:729] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63b66333b164e069820aa416001d92e, State: Initialized, Role: FOLLOWER
I20260808 10:12:19.425132 25959 consensus_queue.cc:260] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } }
I20260808 10:12:19.425571 25959 sys_catalog.cc:564] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:19.425554 25967 sys_catalog.cc:455] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } } }
I20260808 10:12:19.425671 25967 sys_catalog.cc:458] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e [sys.catalog]: This master's current role is: FOLLOWER
W20260808 10:12:19.427506 25979 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:12:19.427573 25979 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:12:19.427922 25887 master_runner.cc:186] Error getting master registration for 127.2.30.190:41027: OK, instance_id { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" instance_seqno: 1786183939022431 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:12:19.430126 25887 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:12:19.454473 25879 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } attempt: 3
Aug 08 10:12:19 dist-test-slave-w206 krb5kdc[25715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183939, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.189@KRBTEST.COM for kudu/127.2.30.188@KRBTEST.COM
I20260808 10:12:19.462831 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.454941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:34277 (local address 127.0.0.1:59954)
0808 10:12:19.455038 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.455053 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:19.455122 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:19.455288 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:19.455289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:19.455560 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:19.455611 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.455613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.456321 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.456322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.457269 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.457272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.457678 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.457679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.457739 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.458257 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:19.458263 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:19.459796 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:19.461664 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.461665 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.461665 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.461858 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.462079 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.462080 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.462080 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.462147 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.462469 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:19.462470 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:19.462520 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 10:12:19.462611 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 10:12:19.462724 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 10:12:19.462831 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.455070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59954 (local address 127.2.30.188:34277)
0808 10:12:19.455120 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.455122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.455134 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:12:19.455187 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.455188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.455200 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.455266 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.455713 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.456226 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.457396 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.457597 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.457680 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.459933 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:19.459934 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:19.459935 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:19.459944 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:19.461543 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.461938 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.461939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.461939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.462005 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.462212 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.462213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.462213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.462394 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:19.462478 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:19.462635 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 10:12:19.462724 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 10:12:19.464689 25879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476: Bootstrap starting.
I20260808 10:12:19.465373 25879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:19.465677 25879 log.cc:824] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:19.466481 25879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476: No bootstrap required, opened a new log
I20260808 10:12:19.468082 25879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } }
I20260808 10:12:19.468235 25879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23a8b6b318874db89f490656f73f2476, State: Initialized, Role: FOLLOWER
I20260808 10:12:19.468351 25879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } }
I20260808 10:12:19.468611 25801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } attempt: 3
I20260808 10:12:19.468765 25879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:19.468685 25982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } } }
W20260808 10:12:19.470800 25994 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:12:19.470908 25994 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:12:19.470872 25809 master_runner.cc:186] Error getting master registration for 127.2.30.190:41027: OK, instance_id { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" instance_seqno: 1786183939022431 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 10:12:19.473495 25809 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:12:19.473595 25982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476 [sys.catalog]: This master's current role is: FOLLOWER
Aug 08 10:12:19 dist-test-slave-w206 krb5kdc[25715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786183938, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.30.190@KRBTEST.COM for kudu/127.2.30.188@KRBTEST.COM
I20260808 10:12:19.478963 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.468957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.30.188:34277 (local address 127.0.0.1:59966)
0808 10:12:19.469011 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 10:12:19.469023 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 10:12:19.469065 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 10:12:19.469418 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 10:12:19.469419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 10:12:19.469622 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 10:12:19.469694 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.469696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.470759 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.470760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.471629 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 10:12:19.471632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.472189 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.472190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 10:12:19.472251 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.472870 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 10:12:19.472876 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 10:12:19.474507 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 10:12:19.477764 (+ 3257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.477766 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.477766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.478020 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.478252 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 10:12:19.478253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 10:12:19.478253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 10:12:19.478314 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 10:12:19.478625 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 10:12:19.478626 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 10:12:19.478687 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 10:12:19.478736 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 10:12:19.478854 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 10:12:19.479457 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.469205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59966 (local address 127.2.30.188:34277)
0808 10:12:19.469277 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.469278 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.469286 (+ 8us) server_negotiation.cc:407] Connection header received
0808 10:12:19.469314 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.469315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.469326 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.469768 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.469823 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.470633 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.471803 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.472096 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.472460 (+ 364us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.474626 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:19.474628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:19.474628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:19.474637 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:19.477614 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.478114 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.478116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.478116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.478176 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.478383 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.478384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.478384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.478547 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:19.479228 (+ 681us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:19.479272 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 10:12:19.479364 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 10:12:19.481217 25801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6: Bootstrap starting.
I20260808 10:12:19.482040 25801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6: Neither blocks nor log segments found. Creating new log.
I20260808 10:12:19.482366 25801 log.cc:824] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6: Log is configured to *not* fsync() on all Append() calls
I20260808 10:12:19.483367 25801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6: No bootstrap required, opened a new log
I20260808 10:12:19.484874 25801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } }
I20260808 10:12:19.485072 25801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e1b7670c9ec4e089203c4284a16e7a6, State: Initialized, Role: FOLLOWER
I20260808 10:12:19.485179 25801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } }
I20260808 10:12:19.485569 25801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 10:12:19.485540 25997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } } }
I20260808 10:12:19.485749 25997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:12:19.487062 25773 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 10:12:19.487164 25773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:19.488332 25773 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } }
I20260808 10:12:19.488701 25773 leader_election.cc:288] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [CANDIDATE]: Term 1 election: Requested vote from peers 23a8b6b318874db89f490656f73f2476 (127.2.30.189:33359), d63b66333b164e069820aa416001d92e (127.2.30.188:34277)
W20260808 10:12:19.489730 26008 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 10:12:19.489784 26008 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 10:12:19.489985 25852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "23a8b6b318874db89f490656f73f2476"
I20260808 10:12:19.490150 25852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476 [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:19.491367 25852 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e1b7670c9ec4e089203c4284a16e7a6 in term 1.
I20260808 10:12:19.491667 25930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d63b66333b164e069820aa416001d92e"
I20260808 10:12:19.491830 25930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e [term 0 FOLLOWER]: Advancing to term 1
I20260808 10:12:19.492841 25930 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e1b7670c9ec4e089203c4284a16e7a6 in term 1.
I20260808 10:12:19.493568 25746 leader_election.cc:302] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23a8b6b318874db89f490656f73f2476, 8e1b7670c9ec4e089203c4284a16e7a6; no voters:
I20260808 10:12:19.493826 25997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 10:12:19.493969 25997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [term 1 LEADER]: Becoming Leader. State: Replica: 8e1b7670c9ec4e089203c4284a16e7a6, State: Running, Role: LEADER
I20260808 10:12:19.494061 25997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } }
I20260808 10:12:19.494561 25997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e1b7670c9ec4e089203c4284a16e7a6. Latest consensus state: current_term: 1 leader_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } } }
I20260808 10:12:19.494642 25997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:19.497138 25728 master_runner.cc:263] No leader master found from master 8e1b7670c9ec4e089203c4284a16e7a6
I20260808 10:12:19.497202 25728 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 10:12:19.504011 26016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 10:12:19.504379 26016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 10:12:19.519366 25804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.503310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47174 (local address 127.2.30.190:41027)
0808 10:12:19.503362 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.503365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.505417 (+ 2052us) server_negotiation.cc:407] Connection header received
0808 10:12:19.505524 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.505525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.505538 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.505569 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.506386 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.507202 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.509376 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.509690 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.509736 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.514430 (+ 4694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:19.514432 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:19.514433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:19.514443 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:19.516581 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.518230 (+ 1649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.518232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.518232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.518306 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.518518 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.518518 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.518519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.518669 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:19.518781 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:19.519096 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 10:12:19.519227 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 10:12:19.520210 25750 tablet.cc:2369] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 10:12:19.520362 25930 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e [term 1 FOLLOWER]: Refusing update from remote peer 8e1b7670c9ec4e089203c4284a16e7a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:12:19.520932 26014 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:12:19.522765 25852 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476 [term 1 FOLLOWER]: Refusing update from remote peer 8e1b7670c9ec4e089203c4284a16e7a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 10:12:19.523092 26014 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 10:12:19.524240 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.503557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59970 (local address 127.2.30.188:34277)
0808 10:12:19.503605 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.503606 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.508415 (+ 4809us) server_negotiation.cc:407] Connection header received
0808 10:12:19.508518 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.508520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.508533 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.508563 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.510364 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.511147 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.513466 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.513802 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.513849 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.520729 (+ 6880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:19.520731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:19.520731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:19.520744 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:19.522689 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.523354 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.523355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.523355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.523418 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.523622 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.523622 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.523623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.523769 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:19.523854 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:19.524052 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 10:12:19.524140 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 10:12:19.524879 25982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e1b7670c9ec4e089203c4284a16e7a6. Latest consensus state: current_term: 1 leader_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } } }
I20260808 10:12:19.524989 25982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:12:19.526479 26014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } } }
I20260808 10:12:19.526582 26014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:19.526649 26020 mvcc.cc:205] Tried to move back new op lower bound from 7316209416271507456 to 7316209416168501248. Current Snapshot: MvccSnapshot[applied={T|T < 7316209416271507456}]
I20260808 10:12:19.527051 25967 sys_catalog.cc:455] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e1b7670c9ec4e089203c4284a16e7a6. Latest consensus state: current_term: 1 leader_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } } }
I20260808 10:12:19.527161 25967 sys_catalog.cc:458] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:12:19.527132 25982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } } }
I20260808 10:12:19.527240 25982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:12:19.527473 25997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } } }
I20260808 10:12:19.527565 25997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6 [sys.catalog]: This master's current role is: LEADER
I20260808 10:12:19.527853 26023 mvcc.cc:205] Tried to move back new op lower bound from 7316209416271507456 to 7316209416168501248. Current Snapshot: MvccSnapshot[applied={T|T < 7316209416271507456}]
I20260808 10:12:19.527992 26021 mvcc.cc:205] Tried to move back new op lower bound from 7316209416271507456 to 7316209416168501248. Current Snapshot: MvccSnapshot[applied={T|T < 7316209416271507456}]
I20260808 10:12:19.528107 25967 sys_catalog.cc:455] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e1b7670c9ec4e089203c4284a16e7a6" member_type: VOTER last_known_addr { host: "127.2.30.190" port: 41027 } } peers { permanent_uuid: "23a8b6b318874db89f490656f73f2476" member_type: VOTER last_known_addr { host: "127.2.30.189" port: 33359 } } peers { permanent_uuid: "d63b66333b164e069820aa416001d92e" member_type: VOTER last_known_addr { host: "127.2.30.188" port: 34277 } } }
I20260808 10:12:19.528193 25967 sys_catalog.cc:458] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e [sys.catalog]: This master's current role is: FOLLOWER
I20260808 10:12:19.531623 26016 catalog_manager.cc:1347] Generated new cluster ID: b4d43cce86ff402a8bfd4725f6126c54
I20260808 10:12:19.531698 26016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 10:12:19.538403 25885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:19.495323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59020 (local address 127.2.30.189:33359)
0808 10:12:19.495373 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.495376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.503680 (+ 8304us) server_negotiation.cc:407] Connection header received
0808 10:12:19.503823 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.503826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.503838 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.503872 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.511117 (+ 7245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.511836 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.527322 (+ 15486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:19.527677 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:19.527728 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:19.529524 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 10:12:19.529527 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 10:12:19.529527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 10:12:19.529535 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 10:12:19.531318 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.537388 (+ 6070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.537390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.537391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.537491 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 10:12:19.537740 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 10:12:19.537741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 10:12:19.537741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 10:12:19.537886 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 10:12:19.537977 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:19.538197 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 10:12:19.538298 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 10:12:19.540161 26016 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 10:12:19.541594 26016 catalog_manager.cc:1504] Loading token signing keys...
I20260808 10:12:19.547652 26016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e1b7670c9ec4e089203c4284a16e7a6: Generated new TSK 0
I20260808 10:12:19.547935 26016 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 10:12:19.580708 25804 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:19.578994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47186 (local address 127.2.30.190:41027)
0808 10:12:19.579040 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.579042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.579889 (+ 847us) server_negotiation.cc:407] Connection header received
0808 10:12:19.580036 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.580037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.580049 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.580083 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.580619 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47186: BlockingRecv error: recv got EOF from 127.0.0.1:47186 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 10:12:19.581921 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:19.578717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59034 (local address 127.2.30.189:33359)
0808 10:12:19.578770 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.578772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.581311 (+ 2539us) server_negotiation.cc:407] Connection header received
0808 10:12:19.581441 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.581443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.581458 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.581491 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.581852 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59034: BlockingRecv error: recv got EOF from 127.0.0.1:59034 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 10:12:19.582832 25963 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:19.579570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59984 (local address 127.2.30.188:34277)
0808 10:12:19.579617 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.579619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.582302 (+ 2683us) server_negotiation.cc:407] Connection header received
0808 10:12:19.582403 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.582405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.582417 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.582448 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.582766 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59984: BlockingRecv error: recv got EOF from 127.0.0.1:59984 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 10:12:19.585639 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:19.584738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59048 (local address 127.2.30.189:33359)
0808 10:12:19.584779 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.584781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.585060 (+ 279us) server_negotiation.cc:407] Connection header received
0808 10:12:19.585186 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.585188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.585210 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.585246 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.585589 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59048: BlockingRecv error: recv got EOF from 127.0.0.1:59048 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 10:12:19.588994 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:19.588103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59052 (local address 127.2.30.189:33359)
0808 10:12:19.588142 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.588144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.588446 (+ 302us) server_negotiation.cc:407] Connection header received
0808 10:12:19.588552 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.588553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.588565 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.588595 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.588941 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59052: BlockingRecv error: recv got EOF from 127.0.0.1:59052 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 10:12:19.593436 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:19.592448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59068 (local address 127.2.30.189:33359)
0808 10:12:19.592492 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.592494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.592801 (+ 307us) server_negotiation.cc:407] Connection header received
0808 10:12:19.592911 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.592912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.592924 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.592955 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.593377 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59068: BlockingRecv error: recv got EOF from 127.0.0.1:59068 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 10:12:19.600205 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:19.599129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59074 (local address 127.2.30.189:33359)
0808 10:12:19.599187 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.599190 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.599528 (+ 338us) server_negotiation.cc:407] Connection header received
0808 10:12:19.599641 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.599642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.599654 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.599688 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.600133 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59074: BlockingRecv error: recv got EOF from 127.0.0.1:59074 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 10:12:19.610800 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:19.609844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59078 (local address 127.2.30.189:33359)
0808 10:12:19.609893 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.609895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.610184 (+ 289us) server_negotiation.cc:407] Connection header received
0808 10:12:19.610293 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.610294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.610306 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.610336 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.610728 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59078: BlockingRecv error: recv got EOF from 127.0.0.1:59078 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 10:12:19.629293 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:19.628490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59082 (local address 127.2.30.189:33359)
0808 10:12:19.628538 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.628540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.628694 (+ 154us) server_negotiation.cc:407] Connection header received
0808 10:12:19.628768 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.628769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.628781 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.628849 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.629212 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59082: BlockingRecv error: recv got EOF from 127.0.0.1:59082 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 10:12:19.663501 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:19.662610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59084 (local address 127.2.30.189:33359)
0808 10:12:19.662668 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.662670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.662836 (+ 166us) server_negotiation.cc:407] Connection header received
0808 10:12:19.662912 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.662913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.662927 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.663032 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.663419 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59084: BlockingRecv error: recv got EOF from 127.0.0.1:59084 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 10:12:19.729679 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:19.728737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59100 (local address 127.2.30.189:33359)
0808 10:12:19.728845 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.728847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.728875 (+ 28us) server_negotiation.cc:407] Connection header received
0808 10:12:19.728945 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.728946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.728957 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.729017 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.729599 (+ 582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59100: BlockingRecv error: recv got EOF from 127.0.0.1:59100 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260808 10:12:19.859786 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:19.859049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59106 (local address 127.2.30.189:33359)
0808 10:12:19.859130 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:19.859134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:19.859150 (+ 16us) server_negotiation.cc:407] Connection header received
0808 10:12:19.859194 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:19.859195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:19.859209 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:19.859282 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:19.859706 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59106: BlockingRecv error: recv got EOF from 127.0.0.1:59106 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 10:12:20.117981 25885 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 10:12:20.117189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59120 (local address 127.2.30.189:33359)
0808 10:12:20.117263 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:20.117265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:20.117328 (+ 63us) server_negotiation.cc:407] Connection header received
0808 10:12:20.117414 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:20.117415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:20.117427 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:20.117501 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 10:12:20.117884 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59120: BlockingRecv error: recv got EOF from 127.0.0.1:59120 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 10:12:20.428066 25979 catalog_manager.cc:1259] Loaded cluster ID: b4d43cce86ff402a8bfd4725f6126c54
I20260808 10:12:20.428189 25979 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e: loading cluster ID for follower catalog manager: success
I20260808 10:12:20.429042 25979 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e: acquiring CA information for follower catalog manager: success
I20260808 10:12:20.429415 25979 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d63b66333b164e069820aa416001d92e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:12:20.471575 25994 catalog_manager.cc:1259] Loaded cluster ID: b4d43cce86ff402a8bfd4725f6126c54
I20260808 10:12:20.471663 25994 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476: loading cluster ID for follower catalog manager: success
I20260808 10:12:20.472544 25994 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476: acquiring CA information for follower catalog manager: success
I20260808 10:12:20.472823 25994 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 23a8b6b318874db89f490656f73f2476: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 10:12:20.634434 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:20.631269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59122 (local address 127.2.30.189:33359)
0808 10:12:20.631441 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:20.631443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:20.631455 (+ 12us) server_negotiation.cc:407] Connection header received
0808 10:12:20.631531 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:20.631531 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:20.631588 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:20.631665 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:12:20.631808 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:20.632349 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:20.633320 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:20.633561 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:20.633640 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:20.633879 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:12:20.634010 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:12:20.634090 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:20.634219 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 10:12:20.634247 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260808 10:12:20.639343 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:20.636626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59996 (local address 127.2.30.188:34277)
0808 10:12:20.636755 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:20.636757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:20.636809 (+ 52us) server_negotiation.cc:407] Connection header received
0808 10:12:20.636897 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:20.636897 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:20.636938 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:20.637014 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:12:20.637136 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:20.637619 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:20.638480 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:20.638670 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:20.638720 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:20.638895 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:12:20.639076 (+ 181us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:12:20.639150 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:20.639201 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 10:12:20.639221 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260808 10:12:20.644599 26096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:20.641345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59138 (local address 127.2.30.189:33359)
0808 10:12:20.641394 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:20.641396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:20.641407 (+ 11us) server_negotiation.cc:407] Connection header received
0808 10:12:20.641443 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:20.641444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:20.641459 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:20.641535 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:12:20.641679 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:20.642505 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:20.643737 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:20.644048 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:20.644121 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:20.644295 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:12:20.644336 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:12:20.644392 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:20.644465 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 10:12:20.644496 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 10:12:20.644975 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:20.641467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60010 (local address 127.2.30.188:34277)
0808 10:12:20.641513 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:20.641515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:20.641522 (+ 7us) server_negotiation.cc:407] Connection header received
0808 10:12:20.641546 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:20.641547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:20.641558 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:20.641611 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:12:20.641754 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:20.642473 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:20.644163 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:20.644458 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:20.644519 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:20.644683 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:12:20.644719 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:12:20.644774 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:20.644879 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 10:12:20.644909 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 10:12:20.645371 26110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 10:12:20.641242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47196 (local address 127.2.30.190:41027)
0808 10:12:20.641542 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0808 10:12:20.641544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 10:12:20.641553 (+ 9us) server_negotiation.cc:407] Connection header received
0808 10:12:20.641600 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 10:12:20.641601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 10:12:20.641661 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 10:12:20.641752 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 10:12:20.642721 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:20.643331 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:20.644324 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 10:12:20.644640 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 10:12:20.644699 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 10:12:20.644923 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 10:12:20.645037 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 10:12:20.645098 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 10:12:20.645271 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 10:12:20.645296 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":54,"threads_started":1}
I20260808 10:12:20.647370 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 25728
I20260808 10:12:20.657526 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 25809
I20260808 10:12:20.664283 2170 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskbFyXgX/build/release/bin/kudu with pid 25887
2026-08-08T10:12:20Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3011 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5990 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (288992 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST